#17 new
James Mead

[#17118] Expectations should take precedence over stubs

Reported by James Mead | January 1st, 2009 @ 07:38 PM

Date: 2008-01-15 16:52 Opener: Nobody

It's a very common practice to setup stubs in a testcase "setup" and set expectations on the same method at the specific tests.

For example:


class FooTest < Test::Unit::TestCase
  def setup
    @foo = Foo.new
    @foo.stubs(:bar).return(true) # common case
  end

  def test_something_shouldnt_call_bar
    @foo.expects(:bar).never
    @foo.something
  end
end

I'd expect this test to fail if @foo.something calls :bar, but in the current version of Mocha (0.5.6) it will succeed.

A test that demonstrate the issue: http://pastie.caboo.se/139119

Comments and changes to this ticket

  • James Mead

    James Mead January 1st, 2009 @ 07:38 PM

    Date: 2008-01-17 13:34 Sender: James Mead

    Just wanted to let you know I am looking into this. Thanks for providing example tests and code.

  • James Mead

    James Mead January 1st, 2009 @ 07:40 PM

    Date: 2008-01-20 17:01 Sender: James Mead

    You are correct that this behaviour did change between Mocha v0.4.0 and v0.5.0 (in revision 115).

    The idea of this change was to bring Mocha into line with jMock v1 "Stubs, Expectations and the Dispatch of Mocked Methods" http://www.jmock.org/jmock1-disp... so that it stops matching expectations when the maximum number of expected invocations is reached. You are suffering from "gotcha" number 2 in that jMock reference: "if you create a stub and then an expectation for the same method, the expectation will match, and when it stops matching the stub will be used instead, possibly masking test failures". In your example, the "never" expectation will never match because it has already reached its maximum number of expected invocations, but Mocha will then find the stub which does match.

    Personally I prefer not to use setup methods, I prefer to make tests as self-contained as possible "Testing: Inline Setup" http://blog.jayfields.com/2007/0... . I find this makes tests more maintainable (albeit at the expense of some duplication). So in this case I would prefer to explicitly put the stub expectation in every test that needed it. If setting up the stub expectation was complex or multiple stub expectations were needed I would extract that into a descriptive method and call it explicitly from each test that needed it.

    If you don't like this approach, you do have another alternative. Using "edge" Mocha (i.e. revision 233 or later), you should be able to use the state-machine feature to achieve your goal OverrideExpectationsFromSetUpAcceptanceTest http://pastie.caboo.se/141188. This should be released as a gem soon.

    Thanks, James. http://blog.floehopper.org

  • James Mead

    James Mead January 1st, 2009 @ 07:41 PM

    Date: 2008-08-06 00:15 Sender: Rick Bradley

    Argh. I've been getting bitten by this too. At least I understand the source of the problem now.

    Just my opinions, but:

    (1) From what I'm reading here, jMock seems broken.

    (2) It seems quite odd to emulate a Java mocking tool in a language as dynamic as Ruby, I've never met a Java testing or mocking tool that I've liked, though, to be fair, that's primarily because of Java as a language.

    (3) I've never bought Jay's argument that no setup is not only a good thing but The Way It Should Be; until now I've not had any reason to care that I was on the other side of that issue.

    (4) The state-based mocking seems like it would promote more bad development practices than prolific use of setup methods -- most time I've seen the need for mocking states, "scenarios", etc., it's masking the fact that the code under test is poorly designed. It SHOULD be painful to write tests for bad code, state-machining relieves the pain of testing poor code.

    (5) Regardless, this was a surprising (and quiet) change to mocha -- existing tests aren't going to break, and TDDing new code exposes this (in a confusing, "wtf, mocha's busted for some reason" way), but characterizing existing code gets a false sense of certainty.

    As far as I can tell, basically this change says "unless you write tests without setup methods, prepare to be surprised or be lulled into a false sense of confidence", without explicitly saying so... except for here, which took me months to find. :-/

    Best, Rick

  • James Mead

    James Mead January 1st, 2009 @ 07:42 PM

    Date: 2008-08-06 00:17 Sender: Yossef Mendelssohn

    I'd like to add my vote to this issue. I'm glad that the state machine feature makes what I want to test possible with a setup method (or more precisely, a before block, since I like RSpec -- and I think Mocha is better than the built-in RSpec mocking), but it's kind of a pain and not entirely nice to work with or look at.

    I understand that you prefer it to work a specific way, that you follow Jay Fields's examples of duplicating code in your tests. What I'm not understanding is

    1. Why it's not possible to write Mocha so that it can be used in a way others expect as well as your preferred style.
    2. Why Mocha needs to be brought into line with jMock v1, gotchas and all.
    3. To take the words out of a colleague's mouth, why you would build a mocking library in a dynamic language modeled after anything written in Java?

    Note that I have answers that may catch your initial responses, or at least point out that I've done some research.

    1. I admit there's probably more to this than just this one example. Constrained to just this, it seems like letting expectations take precedence over stubs would work with the way I (and the reporter, and some colleagues, and unknown others) want to write the tests as well as the way you (and Jay Fields, and unknown others) want to write them. It would also change other behavior that some people (you, Jay Fields, unknown others?) might depend on.
    2. Yes, I know that it's using "a syntax like that of jMock", but that doesn't mean it has to be the Ruby version of jMock. If it does mean that, at least you didn't call it rMock.
    3. Dude, I don't know what to say. I'm not a fan of Java. At all. Especially Java files that have .rb extensions.
  • James Mead

    James Mead January 1st, 2009 @ 07:42 PM

    Date: 2008-08-06 18:54 Sender: James Mead

    Rick: I hope you don't mind, but I've responded to your comment on the mailing list http://groups.google.com/group/m... - I think that's a better forum for discussing the issue.

    Regards, James.

  • James Mead

    James Mead January 1st, 2009 @ 07:43 PM

    Date: 2008-08-06 21:05 Sender: James Mead

    Yossef: I have responded to your comment on the same thread http://groups.google.com/group/m... on the mailing list.

    Regards, James.

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

A mocking & stubbing library for Ruby.

* <a href="http://github.com/floehopper/mocha">GitHub repository</a>
* <a href="http://mocha.rubyforge.org">Documentation</a>
* <a href="http://groups.google.com/group/mocha-developer">Mailing List</a>

People watching this ticket

Tags

Pages