headers
Leslie P. Polzer | 1 Oct 2008 20:19
Picon
Gravatar

Inline tests


Currently the tests are separate from the actual code.
How about inlining?
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "weblocks" group.
To post to this group, send email to weblocks@...
To unsubscribe from this group, send email to weblocks+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/weblocks?hl=en
-~----------~----~----~----~------~----~------~--~---
Permalink | Reply |
headers
Stephen Compall | 1 Oct 2008 20:49
Picon

Re: Inline tests


"Leslie P. Polzer" <leslie.polzer-hi6Y0CQ0nG0@...> writes:
> Currently the tests are separate from the actual code.
> How about inlining?

I see the problem here as well, but I think a solution that would be
equally effective without requiring quite as much legwork would be to
(in dev only) add a post-load ASDF hook that loads weblocks-test and
runs the weblocks suite.

--

-- 
I write stuff at http://failex.blogspot.com/ now.  But the post
formatter and themes are terrible for sharing code, the primary
content, so it might go away sooner or later.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "weblocks" group.
To post to this group, send email to weblocks@...
To unsubscribe from this group, send email to weblocks+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/weblocks?hl=en
-~----------~----~----~----~------~----~------~--~---
Permalink | Reply |
headers
Leslie P. Polzer | 2 Oct 2008 09:26
Picon

Re: Inline tests


> "Leslie P. Polzer" <leslie.polzer-hi6Y0CQ0nG0@...> writes:
>> Currently the tests are separate from the actual code.
>> How about inlining?
>
> I see the problem here as well, but I think a solution that would be
> equally effective without requiring quite as much legwork would be to
> (in dev only) add a post-load ASDF hook that loads weblocks-test and
> runs the weblocks suite.

I'm not sure whether we talk about the same problem here.

My main incentive for inlining the tests would be that they
are right next to the code, thus encouraging contributors
to write new ones and fix existing ones.

What's your problem? :)

  Leslie

--

-- 
LinkedIn Profile: http://www.linkedin.com/in/polzer
Xing Profile: https://www.xing.com/profile/LeslieP_Polzer
Blog: http://blog.viridian-project.de/

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "weblocks" group.
To post to this group, send email to weblocks@...
To unsubscribe from this group, send email to weblocks+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/weblocks?hl=en
(Continue reading)

Permalink | Reply |
headers
Stephen Compall | 2 Oct 2008 21:14
Picon

Re: Inline tests


"Leslie P. Polzer"
<sky-A/mCt7huS1RhCjiJDEAuBxvVK+yQ3ZXh@...> writes:
> My main incentive for inlining the tests would be that they
> are right next to the code, thus encouraging contributors
> to write new ones and fix existing ones.
>
> What's your problem? :)

I wrote about it a little at
http://failex.blogspot.com/2008/09/fixing-weblocks-test-failures-try-1.html

The underlying problem is not that the tests are hard to change; it's
that there is little incentive to run the tests.  Running the tests is
the natural step 1 of changing the tests, and at the moment, checking
for failures is a set-difference operation (aka what failed before my
changes, and what failed after?)

For example, when merged onto weblocks-test-fixes to unmask some
failures that would otherwise be masked by failures before test-fixes,
r836 causes these additional failures:

Failure do-dialog-3
Failure render-widget-body-flash-1
Failure render-widget-body-flash-4
Failure render-widget-body-gridedit-1
Failure render-widget-body-gridedit-3
Failure render-widget-body-gridedit-7

Even if committing onto test-fixes, you'd be hard-pressed to notice this
(Continue reading)

Permalink | Reply |
headers
Leslie P. Polzer | 3 Oct 2008 11:49
Picon

Re: Inline tests


> The underlying problem is not that the tests are hard to change; it's
> that there is little incentive to run the tests.  Running the tests is
> the natural step 1 of changing the tests, and at the moment, checking
> for failures is a set-difference operation (aka what failed before my
> changes, and what failed after?)

You're right.

Sidetracking a bit, how about automating this:

> (TEST-AND-SAVE)
[...]
;; hack, hack, hack
> (TEST-AND-COMPARE)
[..]
You fixed F tests and broke B tests.

> In addition, tests can only encourage writing of new ones when the
> existing failure count is 0.  Right now, any contributor would think
> "the testsuite is broken, just running it will suck me into a bunch of
> set comparisons, hell I could be breaking existing failures more and
> having the failures mask my breakage, and I have to run it before trying
> to add a test, what's the point?"

Count me into that crowd...

> To be fair, I haven't committed to test-fixes in a while; work projects
> have taken more of my time.  However, one project is winding down, and I
> hope to get some interval time to finish it.
(Continue reading)

Permalink | Reply |
headers
Vyacheslav Akhmechet | 5 Oct 2008 21:07
Picon
Gravatar

Re: Inline tests


On Fri, Oct 3, 2008 at 5:49 AM, Leslie P. Polzer
<sky@...> wrote:
> I wonder when the test framework successor will be ready?
I expected the PhD program to leave time for me to work on things, but
I was badly mistaken. Especially time consuming is one of the classes
that mostly involves hacking the Linux kernel [weblocks is a golden
standard of clarity and documentation compared to that :)].

The new test framework has a UI that's almost fully compatible with
Lift. Once I check it in, I'll go through existing tests and port them
(it'll be a matter of running an Emacs macro), so don't let this stop
you. It will just be faster, simpler, consume less memory, and
hopefully be less buggy and more predictable.

> Currently the tests are separate from the actual code.
> How about inlining?
I dislike this idea for two reasons:

- This means the test code will always be loaded along with the
framework, even in a production environment. Of course it's possible
to do some macro hackery to have them expand into NILs in production,
but IMO that makes things less clear.

- Having tests interleaved with source code will (IMO) make code
harder to read and navigate. Of course you could make an opposite
argument, since having unit tests next to the code would act as
additional documentation. What do you think?

I think a better solution would be an emacs function that
(Continue reading)

Permalink | Reply |
headers
Leslie P. Polzer | 6 Oct 2008 10:15
Picon
Gravatar

Re: Inline tests


> - Having tests interleaved with source code will (IMO) make code
> harder to read and navigate. Of course you could make an opposite
> argument, since having unit tests next to the code would act as
> additional documentation. What do you think?

Yeah, it's a two-edged sword, really. I hoped to get some
useful ideas here but it seems there really isn't a panacea.

Let's leave it as it is.

> I think a better solution would be an emacs function that
> automagically switches from source code to a relevant unit test, and
> back (bound to a shortcut, of course). This is also how ruby people do
> it. Do you think this would work?

Not for non-Emacs users.

Not to flame or anything, but I find it really surprising how Emacs
users often like to stay in their editor in a mental sense, too. ;)

But yes, adding this would help at least the Emacs users, so if it's
easy to write an Elisp snippet let's do it.

> How many tests are broken now? Removing (admittedly stupid) HTML
> comparisons from the tests made them much easier to modify and fix. Is
> there a reason tests remain broken now? Does it make sense to abandon
> idealism, accept that the bar is unlikely to be green most of the
> time, and implement TEST-AND-SAVE/TEST-AND-COMPARE paradigm? Or would
> a little more energy fixing the test suite give incentive to people to
(Continue reading)

Permalink | Reply |
headers
Stephen Compall | 6 Oct 2008 17:30
Picon

Re: Inline tests


"Leslie P. Polzer" <leslie.polzer-hi6Y0CQ0nG0@...> writes:
> Not to flame or anything, but I find it really surprising how Emacs
> users often like to stay in their editor in a mental sense, too. ;)

Not to go too far off-topic, but it's pretty comfy here ;)

> But yes, adding this would help at least the Emacs users, so if it's
> easy to write an Elisp snippet let's do it.

I've just been using TAGS.  But yes,

(defun weblocks-toggle-test-file ()
  "If in a source file, switch to the test file, and vice versa."
  (interactive)
  (when buffer-file-name
    (let (src-head src-tail test-head test-tail
          go-src? go-test?)
      (when (string-match "\\(.*\\)/src/\\(.*\\)" buffer-file-name)
        (setq src-head (match-string 1 buffer-file-name)
              src-tail (match-string 2 buffer-file-name)))
      (when (string-match "\\(.*\\)/test/\\(.*\\)" buffer-file-name)
        (setq test-head (match-string 1 buffer-file-name)
              test-tail (match-string 2 buffer-file-name)))
      (cond (src-tail
             (if (and test-tail (> (length src-tail) (length test-tail)))
                 (setq go-src? t)
               (setq go-test? t)))
            (test-tail
             (setq go-src? t)))
(Continue reading)

Permalink | Reply |
headers
Stephen Compall | 6 Oct 2008 02:55
Picon

Re: Inline tests


"Vyacheslav Akhmechet"
<coffeemug-Re5JQEeQqe8AvxtiuMwx3w@...> writes:
> How many tests are broken now?

There are now 4 failures and 3 errors in test-fixes, a branch of r828 if
you cherry r834..835.  Revisions since have added more:

http://www.bitbucket.org/S11001001/weblocks-test-fixes/wiki/LIFT_Failures

The current count is around 25 (as of r887).

--

-- 
I write stuff at http://failex.blogspot.com/ now.  But the post
formatter and themes are terrible for sharing code, the primary
content, so it might go away sooner or later.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "weblocks" group.
To post to this group, send email to weblocks@...
To unsubscribe from this group, send email to weblocks+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/weblocks?hl=en
-~----------~----~----~----~------~----~------~--~---
Permalink | Reply |
headers
Stephen Compall | 4 Oct 2008 00:05
Picon

Re: Inline tests


"Leslie P. Polzer"
<sky-A/mCt7huS1RhCjiJDEAuBxvVK+yQ3ZXh@...> writes:
>> (TEST-AND-SAVE)
> [...]
> ;; hack, hack, hack
>> (TEST-AND-COMPARE)
> [..]
> You fixed F tests and broke B tests.

Yes, this would be nice.

> I wonder when the test framework successor will be ready?

By all means, don't wait on it; Slava mentioned that YACLUTF will be
mostly LIFT-compatible.

http://groups.google.com/group/weblocks/msg/debe9eec6345379e

--

-- 
I write stuff at http://failex.blogspot.com/ now.  But the post
formatter and themes are terrible for sharing code, the primary
content, so it might go away sooner or later.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "weblocks" group.
To post to this group, send email to weblocks@...
To unsubscribe from this group, send email to weblocks+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/weblocks?hl=en
-~----------~----~----~----~------~----~------~--~---
(Continue reading)

Permalink | Reply |

Gmane