185

185 RR Rails 4 Test Prescriptions with Noel Rappin


02:17 – Noel Rappin Introduction

05:46 – Thinking About the Bigger Picture

07:45 – Pair Programming

09:17 – Testing

Tests as Artifacts

17:27 – Deleting Tests

18:24 – Naming and Giving Direction

  • Be Literal

21:02 – Mocking

31:00 – Fixtures

32:26 – JavaScript Testing

35:12 – Testing in the Ruby World vs JavaScript World and Where It’s Heading

37:50 – Time Spent Creating and Refining Tests vs Implementing Tests

40:20 – Creating Data in the Database

42:43 – High Value TDD Tests That Might Not Be As Valuable When Done

  • One Assertion Per Test
  • RSpec

45:10 – Long-Term Regression Tests

46:35 – Test Times and Test Suites

51:33 – Designing and Enforcing Good Testing Practices

54:41 – What Not to Test

56:23 – “Test Smell”

59:18 – Looping in Tests

This episode is sponsored by

comments powered by Disqus

TRANSCRIPT

JESSICA:  A phone with no charger. Wow. David, you are not prepared.

[This episode is sponsored by Hired.com. Every week on Hired, they run an auction where over a thousand tech companies in San Francisco, New York, and L.A. bid on Ruby developers, providing them with salary and equity upfront. The average Ruby developer gets an average of 5 to 15 introductory offers and an average salary offer of $130,000 a year. Users can either accept an offer and go right into interviewing with the company or deny them without any continuing obligations. It’s totally free for users. And when you’re hired, they also give you a $2,000 signing bonus as a thank you for using them. But if you use the Ruby Rogues link, you’ll get a $4,000 bonus instead. Finally, if you’re not looking for a job and know someone who is, you can refer them to Hired and get a $1,337 bonus if they accept a job. Go sign up at Hired.com/RubyRogues.]

[This episode is sponsored by Codeship.com. Don’t you wish you could simply deploy your code every time your tests pass? Wouldn’t it be nice if it were tied into a nice continuous integration system? That’s Codeship. They run your code. If all your tests pass, they deploy your code automatically. For fuss-free continuous delivery, check them out at Codeship.com, continuous delivery made simple.]

[This episode is sponsored by Rackspace. Are you looking for a place to host your latest creation? Want terrific support, high performance all backed by the largest open source cloud? What if you could try it for free? Try out Rackspace at RubyRogues.com/Rackspace and get a $300 credit over six months. That’s $50 per month at RubyRogues.com/Rackspace.]

[Snap is a hosted CI and continuous delivery that is simple and intuitive. Snap’s deployment pipelines deliver fast feedback and can push healthy builds to multiple environments automatically or on demand. Snap integrates deeply with GitHub and has great support for different languages, data stores, and testing frameworks. Snap deploys your application to cloud services like Heroku, Digital Ocean, AWS, and many more. Try Snap for free. Sign up at SnapCI.com/RubyRogues.]

CHUCK:  Hey everybody and welcome to episode 185 of the Ruby Rogues Podcast. This week on our panel, we have Jessica Kerr.

JESSICA: Good morning.

CHUCK:  Coraline Ada Ehmke.

CORALINE:  Hello.

CHUCK:  I’m Charles Max Wood from DevChat.TV. This week we have a special guest and that’s Noel Rappin.

NOEL:  Hey everybody.

CHUCK:  So this week, we’re going to be talking about ‘Rails 4 Test Prescriptions’. It’s not a book club episode but we are going to be talking about the stuff from Noel’s book ‘Rails 4 Test Prescriptions’. Do you want to give us a quick overview on the basic ideas behind the book, real quick?

NOEL:  Yeah. It’s a book about test-driven development specifically for Rails. It starts off by walking you through the beginnings of a sample application doing TDD with Rails. So, we start off with… well actually, we start off with just some basic unit tests. And then eventually we pull in outside-in tests and pull in a full outside-in stream with Capybara and then RSpec. And then from there it breaks down individual pieces. We talk about how to test models, Rails models, and how to test Rails controllers and display logic. And then we talk some more about integration tests. And then we do some more specialized stuff. We talk about minitest. We talked about testing for security. We talk about JavaScript testing. It talks about testing legacy code. It talks about testing external services, how to set up when you’re testing an external service. So, a lot of those things.

The focus is really on practical advice. The focus is not so much on the intricacies of Rails but on testing the kinds of things that come up in web applications rather than really… obviously some of it is dependent on Rails internals and testing some of the Rails specifics. But the idea is that the book will have some really good general advice for testing web applications and for using RSpec and for testing in situations where testing, where people sometimes bail on TDD because they think it’s going to be tricky. And it also tries to make the case for TDD as a really practical, pragmatic, not pie in the sky way of approaching development.

CHUCK:  So, our friend DHH would call this what, a fad diet book?

NOEL:  [Chuckles] We got there. Okay, we’re less than what, three minutes in and we’re already there?

CHUCK:  Yeah, I had to go there.

NOEL:  Yeah. We went there immediately. Yeah, I guess he would. That’s fine. We actually have an email thread with Andy Hunt, publisher, where we toyed with the idea of asking him for a blurb quote.

CHUCK:  [Laughs]

JESSICA:  Or an introduction, maybe.

NOEL:  Yeah, no. I don’t think an introduction was ever in play. But a blurb quote would have been interesting.

[Chuckles]

NOEL:  Yeah, I think that, I haven’t actually spoken to DHH about it but it seems pretty clear we have some differences of opinion about what is practical and most effective. I suspect actually, we’re trying to reach the same goal. And one of the things that I do say in the book and try to make a point of is software’s complicated. And there are a lot of roads to being successful. And I try to point out this is a road that works for me. This is how I think you can get it to work for you a little bit better. But I try not to be the, “You need to do this. This is the only way,” because that’s not real and that’s not helpful. I think it’s helpful to say, “This is how I have made this process successful for me. And these are the tools that make it successful.” And also, “This is the mental process I go through when I’m trying to decide what tests to write and what to do next, that makes it successful and makes it a design process.”

JESSICA:  That’s really important. I think a lot of books skimp on the, why would I do this and when, when they get focused on the how.

NOEL:  Yeah. Well, I think I try to address the why pretty early on. And I think for the how, I try to address the how at a different level than you normally see it. Not how to use RSpec, but how do I think about breaking down a problem into something that I can test? How do I think about breaking functionality into testable pieces? How do I think about what data do I need for this test? How do I think about what I’m actually trying to have this test do in a larger concept and how it might affect the design of the program that I’m writing?

JESSICA:  That makes a lot of sense. You just emphasized thinking about eight times. That was one of the questions that I wanted to ask you today, was in the TDD cycle, in the process of write the test, make it pass, refactor, write a test, where do you usually find yourself pausing to think about the bigger picture?

NOEL:  So, it happens at a couple of different points during it. And one of them is right at the beginning. What am I trying to do? What tests do I write next? And that’s really very much a design question about the functionality of the application you’re writing. What does this actually need to do and how can I express what this needs to do in a way that is automatable and repeatable? So, that’s the first part.

The second part is when you’re actually sitting down to write the test, you’re now thinking at a slightly more practical level how is this test going to interact with my application? What classes is it going to use? What methods is it going to use? Do those classes exist? Do those methods exist? What data does it need to create? And all of those are design questions. You’re designing the API of the way your program interacts with your tests. And that actually turns out to be very valuable for designing the API of how pieces of your program deal with each other. Having the test as this universal client that deals with all of your program is actually a very useful design structure.

And then maybe most directly, when you’re refactoring. A lot of times when I’m doing TDD, I will write the test and then I will write the fastest thing, the first thing I can think of to get the test passing, even if I know it’s not the final version. And then you look at that and you say, “Okay, is there duplication here?” Are there lines of code or methods that are overly complex? Is there something that looks like an abstraction that is waiting to be created? And that’s the last part of design in the process. That’s where it becomes an actual driven design process as opposed to just writing tests to verify that the program does what you think it does.

CORALINE:  And Noel, in the case of pair programming, how does that change or not change?

NOEL:  So, pair programming in that context, one really common pair programming structure is to have one person write the test and the other person write the implementation. I guess in that case where it would change, I think the basic processes are there but then you’re doing it in collaboration with the person you’re pairing with. And how you do that collaboration I guess depends on the pair.

But a really common setup there is that you get into competitive ping pong pairing where the person writing the implementation tries to write the stupidest implementation that can pass the test and daring the person writing the test to write another test that breaks the code. And that is a different way to approach the final design, by putting up obvious straw men candidate for the final code and making the pair writing the test come up with something that breaks it.

DAVID:  Hey, are you guys doing a podcast?

[Laughter]

NOEL:  Did you just…? You just happened to be in the neighborhood, did you?

DAVID:  Yeah.

CHUCK:  It’s a beautiful day in the neighborhood.

[Chuckles]

DAVID:  It’s a… Hi. This is Dave Brady, for the one person who doesn’t know me. Well, and for the thousands who don’t care. But anyway…

NOEL:  Hi, Dave.

DAVID:  IT broke my computer last night, so while you guys were starting the call, I was running around trying to find a microphone. So, I’ve been listening for the past minute but I’m on my app.

NOEL:  Really tempted to ask whether you found one.

[Chuckles]

DAVID:  I’m on my iPad which has no charger but I have 92% battery. And I’ve got the screen brightness turned all the way off. So, we’ll see if I can make it to the end of the call. So, I do have a question for Noel though, which is going back to thinking, one of the things that I’m running into, and this is more of a brownfield question where you’re testing something that was maybe written quickly or it’s legacy code. It can be something you wrote five minutes ago or it can be something that the team wrote six months ago. And how do you let your tests guide you into, I’m trying not to turn the question into leading you to the answer that I want. Let me just put it this way. Can your tests tell you when your application is too complex and complicated and stitched together? And can you then use your tests to actually tease things apart?

NOEL:  Well definitely, yes for the first one. I think that tests are a really important guide to knowing if your code is getting too complicated. If it becomes hard to write tests, a lot of people have the situation where they start using testing and maybe they don’t refactor or they don’t know what to refactor. They know that they should use RSpec but they don’t have the experience or a helpful person nearby to guide them through the process. And they get to the point where the code gets a little bit convoluted. And the tests get harder to write, because they need more setup. There are more objects. The objects relate in complicated ways. You need a lot of different objects set up. And then they go, “Oh, these tests are really hard to write. I give up.”

And I think the thing is that when the tests get hard to write in a TDD process, you think of that as evidence that the code is too complicated. So, that’s a strong guide. If you need to create a lot of different objects in order to run a unit test that tests a single method, that’s often a sign that your code is too complicated. Now, what you can do about that is an open question. And it depends on what your setup is. I do not advocate…

DAVID:  Could we ask somebody who maybe wrote a book about testing?

[Laughter]

NOEL:  Sure, if I find somebody.

DAVID:  Alright.

NOEL:  But then it becomes an issue of, alright I’m working in this legacy codebase. What are my goals here? My goal is to deliver value for my clients.

DAVID:  Yeah, that’s fair.

NOEL:  Or to deliver value to the product. That’s what I mean. You may be in a situation where, I’ve been handed this codebase. I’ve got these one or two bugs to fix. It’s not going to get touched again for six months. And it may not be worth it to go in and spread the pieces of the code all over, take it apart like a watch.

DAVID:  Yeah, yeah.

NOEL:  Put it back together. There’s an aspect to that that is a coding level question. And there’s an aspect to that that is a business level question.

DAVID:  Yeah.

NOEL:  Like, what are you trying to do?

DAVID:  Yeah.

NOEL:  And to that extent, I can’t answer it. If you do want to start using the tests to start picking things apart, then I think the thing to do is to organically grow the test coverage and the quality to the code. You draw a line and you’re like, well we’re going to do new code right. And even if that means we’re going to have temporarily really strange breaks in the code because we’re putting the new code in new classes and having them awkwardly be called from the existing classes so that the new classes are pure and TDD and gradually stuff gets moved into that, I think that’s a really effective way to go. And Michael Feathers’ legacy code book has a lot of, which actually doesn’t talk about testing but it has a lot of structures in place and a lot of recipes for gradually separating your new code from the bad code. And you let it happen over time. But that’s a gradual process. And you may not have the time or business reason to go through that whole process.

DAVID:  Yeah. And I like that it’s gradual, because at one end you can be like at a plastic surgeon and, to mix metaphors, you can be in there polishing rivets. But at the other end in the spectrum, you’re like an army field medic and you’ve got to get that soldier back on its feet and out in the field again as quickly as possible.

NOEL:  Right.

DAVID:  And I like that it’s gradual, right? That you can, it’s like, “Oh, we got to get this thing back on its feet. And we got to get it, you know, da-da-da-da. But while we’re here…”

NOEL:  This code needs to go into rehab for a few months.

DAVID:  Yeah. This code needs, let’s just take five minutes and pull this piece out.

NOEL:  Right.

DAVID:  And move it to where it needs to be. And it’s like the Boy Scout rule.

NOEL:  Yeah, I definitely…

DAVID:  [inaudible] just a little better than you found it.

NOEL:  I definitely believe in that. And I think that that’s a really effective way to go to it. You leave every piece of code you touch a little bit better than you found it when you can. And one of the advantages of that is that, especially if you’re on a team and not everybody is sold on TDD or the way that you’re structuring the objects, it gives the other people on the team a chance to see what it’s like and to be in a position of maintaining the new code.

I’ve seen a couple of cases recently where there was a really radical redesign of the code into a very, what we could consider a very clean services model. And then the people who maintain the services model for one reason or another moved away from the project. And the people who were in charge of maintaining it had relatively little, or significantly less understanding about how the thing should be structured. And you lose the benefit of that. The advantage of a gradual process is it lets people get acclimated to it as well.

DAVID:  Yeah. Well and gradual cleanup is something you have to instill at the cultural level. And then you could have people leave and the new people come in. And they’ll clean up. They might clean up in a different way, but hopefully they’ll continue to go, “Yeah, this smells bad. Let’s fix it.”

NOEL:  That’s fine. I don’t think there’s any architecture decision that you’ve made that you could guarantee is future-proof, even if you do the best that you can.

DAVID:  Yeah.

NOEL:  It’s fine that the people six weeks later come in and make different decisions or make decisions responding to new problems, as long as they start doing the same guidelines. One of the things that I talk about with legacy code is it’s important to be respectful of the fact that the code is running in production.

DAVID:  Yeah.

NOEL:  It’s really tempting to go, “Who are those morons who did this?” especially if it was you.

DAVID:  [Laughs]

NOEL:  But you don’t necessarily know what constraints they were under, what time constraints they were under, what knowledge constraints they were under at the time. And I think anybody who ships something, even if it’s the biggest muddiest ball of mud in the world, you have to give that code a little bit of respect just because it is in the wild and running and hopefully providing someone some value. And you need to approach changes to that in that light. Respecting the developer who did that as you hope that the future developers will respect the things that you do to it.

DAVID:  I used a brain hack for that where I used to go, “What the heck were they thinking?” Now, I sit back and I throw my hands up in the air and I say, “This was a triumph of YAGNI.”

[Laughter]

DAVID:  Because I am from the future.

[Laughter]

NOEL:  I hope there’s a whole ceremony around that. Yeah. But yeah, I think actually in the book I advocate just shaking your fists at the heavens for one minute and then getting back to work.

DAVID:  Yeah. Get it out of your system and then move on.

NOEL:  Yeah.

JESSICA:  We have a tendency to value code based on its beauty and how well we can understand it. But really, the only value our code has is when it’s running and people are finding it useful.

NOEL:  Right. And I think to that end, the virtue of test driving code is that it can allow you to get there faster and more stably. Many of us really love the ability to sit and polish our code until it shines. But that’s not necessarily what’s going to be valuable in the context of what we’re trying to do.

CORALINE:  Noel, I like to think working with legacy code as being an archeologist and trying to discover not only the artifact but the context in which the artifacts were created and used. To that end, as you’re writing your tests, what sort of signals or artifacts do you create simply for the future generations of developers who are going to work with your code?

NOEL:  I think one of the biggest ones is naming, naming the tests. In RSpec, naming the spec comments in such a way that it makes it clear what purpose the test is serving, which is actually hard to do. And I wouldn’t say that I always do it perfectly. But in terms of artifacts that come in, in the tests, the tests are to some extent an artifact that describes what the code is supposed to do. So, if you can say this test is checking to see that users can’t see the administrative screen or, user can’t see another user’s profile, something like that, it can be a valuable guidepost.

One of the trickiest things when you’re looking at a legacy codebase that has a bunch of goofy tests is when it’s reasonable to get rid of tests. Because some of those tests, especially if it’s a legacy codebase, some of those tests may not be valuable. Some of those tests may just be getting in the way. And if you don’t have an idea of what the test was trying to do, then it can be very hard to know whether it’s still providing value six months, a year, two years later.

JESSICA:  That’s my favorite part of programming, deleting code.

[Laughter]

JESSICA:  And deleting tests.

NOEL:  It is, yeah.

JESSICA:  It’s funny. How do you do that? When do you?

NOEL:  Yeah, people get really freaked out about deleting tests. And I think that as somebody who writes about this and talks to novice programmers about testing, you have to be really careful with the idea that sometimes tests are deleteable because that’s a sharp knife. And you don’t want people to use that on their fingers rather than the code. But I think that sometimes, in a typical TDD process you will often create, the first test will be just a setup. Like, I’m testing that I can initialize this object and send it data. That’s often a first test. Once other tests come in, that test if often redundant and you can get rid of it. But coming to the code months later and having the confidence to do that, that’s really tricky and hard to do, almost impossible to do in most codebases.

CHUCK:  So, I want to change directions a little bit. We’ve talked a lot about the benefits of testing.

DAVID:  Actually…

CHUCK:  Oh.

DAVID:  Before you change direction, Coraline asked a really good question about the naming and the giving direction. I had a conversation with a coworker this morning about the types of data that you put into your test objects. Like if you’ve got a person object and you want to test, the first name is required. Okay, that’s fine. You can put just, [makes gibberish sounds], as the first name.

NOEL:  If you can spell it.

JESSICA:  [Laughs]

DAVID:  Yeah.

NOEL:  Yeah.

CHUCK:  [Laughs]

NOEL:  Yeah, that went, some weird Unicode characters in that, I think.

DAVID:  That’s was actually an MD5 checksum, yeah.

[Laughter]

DAVID:  We have a test suite here that will inject just string hashes to do some automated testing. But when I’m testing more manually, how do you feel about test data that will say, first name is Anna or first name is Bob?

NOEL:  There are a couple of things that I do that are tricks in that respect. One of them is to be just as stupid literal as possible. So, first name, FirstName, last name, LastName.

DAVID:  Okay.

NOEL:  First name, first, last name, last, on the grounds that when I get the error, what I’m looking for here is an error message when the test fails that is clear.

DAVID:  Yeah, yeah.

NOEL:  So, if the error message is only going to show the data, then I want that data to be valuable. So, that’s one thing I will do that uses that. For a while, I would do things where I would use fictional characters that had relationships. So, I would use Fred and Barney for people that had friend relationships because that had meaning for me. I do that less on teams than I would do in my own project because it’s something that would have meaning for me. But if it’s an admin user then its name is Admin Adminson, that kind of thing.

DAVID:  Yeah.

NOEL:  That’s one way to do it, is just be ridiculously literal in terms of the focus of it so that even if you just see the data and don’t see the attribute or the object around it and the error message, then you still can see where it came from.

DAVID:  Yeah.

CORALINE:  I think that is so important, Noel. I’m really glad to hear you say that. I consider foo in a test to be a code smell.

DAVID:  Yeah.

CHUCK:  [Laughs]

NOEL:  Yeah. To me, again these are the details that we don’t all do as much as we would like to do. But they’re the details that often make the difference between having something be really super valuable and just ordinary valuable.

JESSICA:  I think it’s a question for David. What does foo smell like?

[Laughter]

CORALINE:  So, is there an aesthetic component there?

[Laughter]

JESSICA:  It’s that extra polish that we put into our tests that makes them communicative. Whenever…

NOEL:  And it’s really not… extra polish, to just forestall something, it’s not like I’m advocating going off for hours to come up with the perfect name. I’m saying the extra 10 seconds to name something other than foo is a valuable 10 seconds.

JESSICA:  Chuck, what were you going to redirect us to?

CHUCK:  I’m going to go back to the DHH thing a little bit. He railed a lot on…

JESSICA:  Oh, great. [Chuckles]

NOEL:  Railed.

CHUCK:  Oh yeah, we’re going to go there, huh?

[Laughter]

NOEL:  Rail, yeah.

CHUCK:  Go after my choice of words. He went off a bit on the mock as style of testing code, especially in Rails. And it was…

JESSICA:  The what kind of testing?

CHUCK:  So, in a lot of cases if you look at a test suite on a Rails app, what you’ll find is that a lot of the tests wind up mocking things out. So for example, when you test your controller, you wind up mocking out to some extent the model layer. When you test your views, you just make the assignments for your instance variables in assigns and then you test the view. And there were certain aspects of that he didn’t like. The one thing that I run into with that is that if you isolate a bit of your code, then you’re not testing the boundaries. And so, as you move through, as you’re being intelligent about it, you wind up doing end-to-end tests to test your boundaries. But is there a good way to overcome the shortcomings of mocking things out as you write unit tests?

NOEL:  So, there are a couple of different things that are teased together there. I think one of them is just a question of taste. I think some people don’t like the mock style. I think in that talk and in other places where he’s talked about it, DHH talks about not feeling like integration level tests can get you, does what he needs to do from testing, which is mostly verification. So, he doesn’t choose to mock out the model layer because in general, and I’m putting words into his mouth which is probably a bad idea, but in general he’ll just write a controller test that calls the model for verification purposes. And in that case, he doesn’t need to mock the model test because he’s not in general writing separate TDD level unit tests for the model, which is a perfectly valid and consistent thing to do. It doesn’t give you the design component of it, but I think it’s also possible that DHH doesn’t necessarily need the design component when he’s putting together a Rails application.

I think that in terms of piecing the little bits of unit tests together into end-to-end tests, I think that there’s, what you want, what I find works for me is to have a mix where you have a lot of relatively fast, or very fast unit tests and a smaller number of slow integration tests that go end-to-end, and that being the mix. And where people get in trouble I think is where they start to do things in the middle, complicated tests at the Rails controller tests that do wind up kind of being integration tests and kind of being unit tests. Those tests tend to be relatively slow and relatively fragile in my experience. And that’s a problem.

One thing that I’ve started using to get around some of, not all of the issues with mocks is that RSpec 3 has this whole new feature around verified mocks where you can say, this mock object is a stand-in for a user. If you call a method that the user doesn’t define, that’s an error. You can do that really easily in RSpec. You can give it an instance. You can give it a class. Or you can say this already is a user. Stub this method. But if you call a method that the user doesn’t know about, flag it as an error. And that gets around one of the biggest sources of error. Not all of them, but one of them.

JESSICA:  That comes close to testing the seams.

NOEL:  Yeah, yeah. It at least tests that you are not making a really, really bad mistake.

[Chuckles]

DAVID:  Yeah. Sandi Metz talked about this exact same problem and she was using tests unit. And she ended up building a test double that would stand in. and she used dependency injection instead of mocking. And her source code changed. But the test double didn’t. So, the test continued to pass.

NOEL:  Right. That’s the risk that you’re taking.

DAVID:  Yeah.

NOEL:  And that’s the thing that RSpec verified mock objects protect against. If the API changes and you’re calling a method that no longer exists, RSpec has to error.

DAVID:  That’s perfect.

CHUCK:  Well, there are two parts of this. One is, okay. I’m stubbing out or I’m using a test double that implements a method call by the same name as this other method call. But the other thing is the parameter signature or the return values or things like that that may change as you change the other class. And so, there still can be a mismatch there, even if you verify that it accepts the method or message that you would send to it. And so…

NOEL:  Yeah. I think the RSpec one catches parameter list. But obviously, if the return value changes, it’s not going to catch that. That would be…

CHUCK:  Yeah. And so, it also seems, and sometimes I’ve actually just written integration tests between two classes because it’s simpler for me to define that than to try and figure out what the overall scenario is through the entire app that tells me that…

NOEL:  Yeah.

CHUCK:  It’s exercising that particular case.

NOEL:  Right.

JESSICA:  From the perspective of coming from Java land, this is fascinating, because this RSpec 3 mock sounds like it’s, well for one thing it’s doing what the compiler does in a statically typed language.

NOEL:  Basically, yeah.

JESSICA:  But it’s also in Java, the mocking framework started out super strict and you had to explicitly say what it was going to call or it would throw an error. And that was a pain in the butt because it made our tests super fragile. So, we’ve actually moved the other way toward mocks toward more and more forgiving.

NOEL:  Yeah.

JESSICA:  Back in the day EasyMock was not easy.

NOEL:  So, I guess we’re all meeting in the middle.

JESSICA:  Right, right. We’re finding the balance, yeah.

NOE:  I was talking about tests as a marker for how good your design is. That if the test gets hard to write that it’s possible the code is too coupled. Mock objects are even more sensitive to that. A heavily mock as test structure is super sensitive since you’re basically adding a line of code to the test for every method that gets called out from. So, mock tests get really hard to write very fast as the code becomes more entangled. And sometimes that means that the code needs to change. And sometimes that means that the mock test is too sensitive.

JESSICA:  Yeah, I consider mocks a code smell. And this goes back to something you said earlier that when the testing gets hard, that’s when the tests are helping you most because they’re telling you that something’s hard and you should question that.

NOEL:  Yeah. The last Java project that I got paid to write which was several years ago now, we had this crazy multitier setup where there were database objects that passed data off to a business level object that passed data off to business level/GUI objects that passed data off to GUI objects. The program was nowhere near complicated enough to support this [inaudible] seven tier architecture that the guy had come up with. And one of the consequences of that was that as I came in and I’m like, “Oh, I’m going to start writing tests,” in order to get anything to work you had to create an object at basically each level of that architecture. And that was clearly, it clearly would just keep people from writing tests because just to write a one-line test you had 30 lines of setup.

And that’s the thing. Long before you get to that point you can say, “Hey, these things are a little bit too coupled and maybe there’s a way that we could tease these apart a little bit so that we can test into the seam.” One of the advantages of tests is that they’re this universal client to your code that gets into all the seams in the code. And hopefully then you have a code that has a lot of small methods and a lot of, for lack of a better word, surface area. And that seems to correspond well to code that’s easier to keep maintained over time.

JESSICA:  Do you ever find that having all these tests in all the seams makes it hard to change those seams when you need to, to refactor at a level higher than the class?

NOEL:  Yes. But the question is hard compared to what? What is hard about that? Yes, it’s a very visible kind of complication when you try to change, to refactor class structures and it breaks a lot of tests. On the other hand, the alternative would be refactoring without breaking a lot of tests which would take less time I would imagine, but it seems like that’s problematic, too. I think that one of the advantages in that context of integration level tests and integration level tests are particularly useful in legacy code for this reason, is that integration level tests, full end-to-end tests, don’t really care what the structure of the code is.

JESSICA:  Exactly.

NOEL:  And that makes them great for legacy code where you really do think you’re going to be changing the structure. It makes them lousy for diagnosing where failures happen. But it makes them really good for being robust against internal changes in the code. Everything is a tradeoff, unfortunately. [Chuckles]

JESSICA:  That makes sense. I’m wondering if when you go to do a large refactor or more commonly implement a change in the requirements, implement a feature change, is one of the first steps, which tests should I delete?

NOEL:  Yeah. Well, it’s like if you start doing this, it depends on where you start. If you start from writing new tests and then you start, if it’s a peer refactor and you’re not writing tests or whether you’re writing tests to control the new feature and that’s driving some change in the architecture, you’ll eventually get a bunch of test failures. And you need to triage those. The test is either, this is failing because it’s stuff that I haven’t written, this is failing because the API has changed and I just need to point it to a different direction, or this is failing because this whole piece of code has been thrown on the floor and we’re not using it anymore and I can delete this test.

And one of the advantages of having small, focused tests in that context is that it’s somewhat easier to say what the purpose of the test is and whether that purpose is still valuable. But a large code change is going to be problematic whether or not you have tests. It’s sometimes valuable and sometimes necessary. But if you have tests then the tests do sometimes act as a little bit of a drag on it, especially if they were not written super carefully to begin with. And I think it’s almost impossible to be super careful 100% of the time. So, everybody’s going to have that problem. I certainly do. But at the same time, if you try to do a big code change without tests, then you have the problem of just verifying that you’ve made the change that you think you’ve made

DAVID:  So, I have an odd question for you, Noel.

NOEL:  Okay.

DAVID:  I created a fixture today in a Rails 4 project. How bad a person am I?

NOEL:  Oh, it depends on what you were trying to do. I use fixtures in Rails 4 projects. I still do.

DAVID:  I feel better now.

JESSICA:  I [inaudible] nothing to do with that fixture.

NOEL:  Yeah, that’s true David. Yeah, I wouldn’t want to have the fixture be totally measuring your value. But okay.

DAVID:  [Chuckles]

NOEL:  So, here’s what I use fixtures for in Rails 4 projects. I use them for a couple of different things. One of them is for semi-static data that goes in the database, like we have a list of dates or we have a list of states and they’re in the database. And they’re kind of static but they’re in the database. I’ll put those in fixtures. So, they’re there and I don’t have to worry about them because sometimes the code will just be dependent on it. And I also, and this is kind of a pain in the neck, but I also use them in Cucumber integration tests, too. because the good part of fixtures which is that they’re really fast and allow you to create a lot of objects easily for global data actually play really nicely with end-to-end Cucumber style tests. Like, you have some real data in there and you don’t have to create them all at retest. So, I actually use fixtures for that.

DAVID:  Awesome.

CORALINE:  Do you generate the fixtures from real data and just export it in samples? Or do you create it from scratch?

NOEL:  It depends. I have done both. If there actually is real data that’s valuable then I’ll try to use real data. If it doesn’t really matter very much, then I don’t sweat it too much.

CHUCK:  So, I have another question that I get asked a lot about testing. And that is, how do you master time and space with JavaScript tests?

NOEL:  [Laughs] Well, thank you. For any of you that don’t know, Chuck is helpfully plugging a JavaScript book I wrote called ‘Master Time and Space with JavaScript’. JavaScript testing on the one hand is it’s tricky in part because first of all for a long time the tooling wasn’t as good as the Ruby testing. And for a long time, I mean forever until about a year ago. And also, because of the kinds of things that JavaScript gets used for in most web applications traditionally, the kind of very visual, not a whole lot of logic, click here and something moves kind of things, are not necessarily amenable to the kinds of unit tests that we’re used to writing as Rails developers in Ruby.

There are a couple of things that I like to do when the JavaScript gets more complicated. Obviously if I’m using a framework like Ember then Ember has a testing structure built into it and I use that. If I’m not then one of the things that are nice about JavaScript testing is that if you stub, if you use one of the tools that fakes the server, what we would think of as integration tests in JavaScript are really cheap. You can go speed-wise. You can do an end-to-end test of some JavaScript interaction and it’s not doing anything like a Rails codebase would to hit the database. You just stub the server to feed it some sample data. And it’s much, much faster relative to unit tests than it is in the server-side world, at least in my experience. So, I do a fair amount of that.

Another thing that I’ve talked about trying to do as the JavaScript gets more complicated is to treat the DOM and the page, at least think about treating the DOM as a third-party dependency the same way that in Rails we try to encourage people to think of the database as a third-party dependency, so to write code that interacts with the DOM only along fairly constrained places. So, you have objects that essentially to treat the DOM as a service, as an external service that you have an adaptor between your code and the DOM. And that way, that makes it really easy to test your code, your logic in isolation. And then you can integration test to make sure that the DOM changes are happening. That becomes valuable at a certain complexity level of the client-side JavaScript.

So, it doesn’t, it wouldn’t necessarily be the first tool I’d point to if it’s just, “Hey, I want somebody to click on this and I want this to vanish.” But it’s a useful way to think about how to structure JavaScript code. And I’m not super up on all of the… One of the things about JavaScript, the JavaScript testing world is kind of fragmented in terms of the number of tools relative to the Rails words. And I have my favorites and the ones I’m used to. And I’m starting to explore some of the ones that I haven’t, some of the newer ones that I haven’t used as much.

CORALINE:  We talked about the difference between, Jessica brought up the difference between testing in the Java world and testing in the Ruby world. What direction do you feel JavaScript testing is going into?

NOEL:  Well, I think it probably started. I think it’s probably heading in more or less the same arc as the Ruby test is, which is that it started very wide open and it’s probably going to start to clamp down a little bit. JavaScript in some ways is even more, a more open and flexible language than Ruby is. And the testing packages reflect that. It’s, writing a mock object package in JavaScript is in some ways really straightforward because JavaScript has function objects you could just wrap. You can create function objects relatively easily, which makes them potentially very, very wide open. I can’t think of a JavaScript… to do in JavaScript that kind of validation testing that the RSpec thing is doing, I think would be really tricky. But I’d be interested to see if somebody tries to do it.

I think that what will happen over time on JavaScript is that people will get more and more invested in whatever testing solution the frameworks wind up providing, because I think more and more the complicated JavaScript is going to move into Angular. It’s going to move into Ember. It’ll move into React plus whatever people are using to put a [super] structure on React. And I think all of those things are going to increasingly, they already are and I think they’ll increasingly provide their own higher level testing tools.

JESSICA:  I noticed that in the Java world too, that the better frameworks now provide testing frameworks and tools of their own.

NOEL:  Yeah. And I think that that becomes part of the expectation of it. I think that one of the things that the Rails core team deserves a lot of credit for is that from the very beginning, testing was super baked into the way Rails approached the world. And it may not have turned out to be the kind of testing that everybody who used Rails forever wanted to use.

But Rails came down with a really, really strong perspective which was really one of the more new… One of the exciting things about Rails when I first came to Rails was that it had this kind of stuff built in, that you had the concept of a test database separate from your development database, that you had these things called fixtures that let you easily set up data, that there was this whole infrastructure around being able to really easily set up and run tests against a Rails application. And that’s been baked into Rails from day one. And it’s been a really critical piece of why Rails has wound up successful.

JESSICA:  Wow. That’s very cool. It feels to me like lately languages have been optimized to the point that writing the implementation is the easy part. And being sure that it works is the hard part. Noel, how much time do you spend creating and refining tests compared to implementing in the production code?

NOEL:  It depends a little bit on how business logic-y the thing that I’m doing is. I think that one of the points that DHH made that I partially agree with but don’t completely agree with when he said that a lot of Rails applications are basically just moving information around and don’t have a lot of deep logic and don’t need a lot of tests. I think that’s kind of true. There are certainly applications that I write that are doing basically boilerplate kind of stuff where I don’t spend a lot of time making sure I get the tests right because the framework handles a lot of it. If I’m doing something…

I’ve worked on codebases that have fairly complicated logic to manage like a ticketing system or something like that. And in that case, you really do want to make sure you’re structuring the tests to get all of the possible cases. And to set it up so that if somebody, if you see behavior that’s surprising or something new comes up then it’s pretty easy to set up a new case to exercise the behavior you’re seeing or the bug you’re seeing, or the new feature that’s being developed. And in that case, where the underlying business logic gets more complicated, then I do try to spend more time trying to figure out, what kind of tests do I want here? How many tests do I want here? How can I make sure that this is going to provide both the function of using the test to drive the design and the function of using the test to verify that this code continues to work?

Because one of the things that gets overlooked in TDD is this double life that tests have in a TDD process, that we write them once and run them once as part of the code being created. But once the code is written, the test continues to live and it runs hundreds and thousands of times forever. And those two kinds of tests have slightly different needs. The things that you do when you’re writing tests specifically for domain and business logic discovery are not necessarily the kinds of things that are valuable when a test is just running for verification.

JESSICA:  Exactly. Do you ever go back and delete the ones that were really only useful to drive the design?

NOEL:  Sometimes. Again, people get nervous when you start deleting tests. Another thing that’s valuable is to just look at, RSpec has a profiler that will just tell you where the slow test, what the slowest tests are. And sometimes it’s just valuable to look at the slowest, the very slowest tests and try and figure out what’s going on and whether the test is still valuable, why it’s slow, and whether it needs to be… invariably, or almost certainly in a Rails application it’s going to be slow because it’s creating a lot of data. And then the question is just like, is there a better way to do this? So, that can be a valuable thing to do sometimes.

CORALINE:  Noel, do you believe in creating data in the database as part of your testing process? Because that is something that leads to long-running tests in the future.

NOEL:  Right. So, that’s a big issue. One of the tricks that Rails did when it created testing was it initially set up things so that putting objects in the database is a major piece of the way Rails set up unit testing from the get go. And Rails set up this idea where we were all lulled into the belief that a unit test could touch the database even though the database is clearly a massive third-party dependency. So, you have people like Corey Haines and Gary Bernhardt and many other people writing Rails tests in such a way that they don’t touch the database, which involves structuring your Rails application in such a way that you can manage business logic without touching the database, whether that means writing tests that have stubs or separating your business logic from the Active Record class so that you can test the business logic separately. Both of those have value.

I think that sometimes you do need to create data in the database. If you’re specifically testing whether you’re writing an Active Record finder that should find one object in the database and not another in the database, it’s really hard to do that without putting objects in the database. So, I try to avoid it. But sometimes it’s so much easier for development to create one object, one or two objects, and put them in a test that [I do it].

And really, the scale of most Rails applications, having tests that put one or two objects in the database is not the long-term problem. The long-term problem is you have tests that create entire object trees of stuff and create dozens of objects. And that’s really the thing you need to watch out for. If I can write a test without putting an object in the database, I will. If I can’t, I spend some time trying to think about why I can’t and whether that means that the code needs to be restructured. But sometimes it just means that the database is an [integral] part of this functionality and it needs to be there.

CHUCK:  So, David asked in the chatroom. He hates moving conference rooms. So, yeah [inaudible].

[Chuckles]

CHUCK:  Exactly.

NOEL:  I’m picturing David in some sort of dark hell-scape where microphones don’t work and people are just randomly booting…

JESSICA:  [Laughs]

NOEL:  I’m picturing him in almost like a first-person shooter looking for microscopes.

JESSICA:  [Laughs]

NOEL:  And running around a maze and trying to find the secret door that has the conference room that he can actually talk in.

CHUCK:  [Laughs]

JESSICA:  He’s laughing. We can’t hear it, but he’s laughing.

CHUCK:  It’s like a programmer Doom or something?

NOEL:  Yeah, yes [Chuckles].

JESSICA:  Yeah.

NOEL:  Right, Dave’s saying that his iPad battery is probably representing his health.

CHUCK:  But anyway, so he asked if you can give us a good example of a high-value TDD test that might not be as available after development is done.

NOEL:  So, a couple of things about that. The speed of a test is much more valuable in a long-term regression test than it is in development. You’re willing to put up with maybe a little bit of a slower test for the couple of times, because sometimes, a lot of times especially on a large Rails suite just as a practical matter you only run the focus tests that you’re worried about initially as you’re writing the code. So, if that one file is two seconds instead of a half a second it’s not that big a deal. But when you’re running the whole test suite and every single one of them is two seconds instead of a half a second, then that becomes a problem.

So, one thing that is sometimes valuable in the design part that is less valuable in the regression part is having one assertion per test. So, there’s a style where you do a lot of stuff in the setup and then you have very, very small focus tests that each make one assertion. And the setup gets run multiple times, which is really great in development because it means that all your assertions get run as opposed to a typical RSpec structure where the first assertion that fails ends the test and you can’t see whether the ones after that have passed or failed. But that adds potentially a lot of overhead because you’re running the same setup multiple times.

So, one thing that’s a valid thing to do is to have a one assertion per test setup as you’re writing the code. Once you’re convinced that the code works and the tests pass, you convert that back to a single test with multiple assertions which runs faster over the long haul. So, that’s one thing that I actually do that’s a distinction between test-driven and the long haul of having tests live in the codebase over eternity.

JESSICA:  That is a really useful distinction. The other thing that you can sometimes get from the longer test with multiple assertions is a story. Sometimes, those are easier to read than a dozen.

NOEL:  Yeah, yeah, I think that’s true. Another thing that RSpec 3 has that could be a nice middle ground here is RSpec 3 has these really interesting compound assertions where you can actually assert an entire complex data structure in one shot. And actually, in the right circumstances it’s actually pretty readable. And then it gives you both the benefit of the speed of only running the setup once and the benefit of checking all of the things at once and showing you all of the problems in a complex data structure. So, it’s something that the RSpec core team seems to really be pushing in terms of being a happy middle ground between those two things.

CHUCK:  And then the other part of it was a high-value long-lived documentation regression test. And you kind of talked a little bit about that, too.

NOEL:  Yeah. I think the valuable things for long-term regression tests are speed and naming and things like that that are going to be valuable for helping you pick up context when you don’t have context. When you’re in the TDD, the actual TDD structure, you have the context. Almost by definition you’re working on the code right then. So, some of the documentation things are less valuable upfront.

JESSICA:  Very true. I find the same problem with refactoring. Sometimes I don’t see a need to refactor something because it’s completely obvious to me after I just wrote it. And then I come back to it later and I have to struggle to figure out what this means. And then that becomes the time to refactor.

NOEL:  Right, yeah. I think that that makes a lot of sense. And then by that point, you almost need to treat your existing code, your own code, as though it was legacy code because you started to lose the context under which it was developed. And to me that’s the definition of legacy code. It’s code that you don’t have the context under which it was written. And you start to lose that almost immediately. [Chuckles] Your context depreciates as soon as you drive the code off the lot, I guess.

CHUCK:  [Chuckles]

NOEL:  And so, coming back to it that can be a great time to refactor. But you may not have all of the context. And again, that can be a good thing, too.

JESSICA:  If you trust your tests, then reading code, also the Boy Scout rule applies. You can improve it as you’re reading.

NOEL:  Yeah.

CORALINE:  Noel, we’ve talked a lot about legacy code and we started touching on test times. And I just have to bring up the fact that when you end up with a long in the tooth Rails application it’s not unusual in my experience for local testing not to happen at all and throwing everything over the wall to CI.

NOEL:  Yeah.

CORALINE:  And that of course ruins the whole point of testing as a feedback loop as part of your design and development. So, what are some strategies [inaudible] that?

NOEL:  The best strategy for dealing with that is not to get in that situation in the first place. But of course, once you’re in that situation that’s horrible advice.

CORALINE:  Barring time travel, yes.

NOEL:  Yeah, barring time travel. Yeah, I worked at a place and I wasn’t a direct developer on that at this place that I’m talking about. But I certainly worked at a place that had all of their testing happen on a very, very powerful CI server farm and it took 30, 40 minutes on the server farm. Nobody had any idea how long it took on a local laptop because nobody in their right mind would ever try to run the test suite on their local machine.

CORALINE:  How do you get out of that?

NOEL:  Again, I think it’s, there are a couple of things. You can do the quick triage which is to find the slowest test and try to fix the slowest test. At the place that I’m referring to, they actually did that for a week. They had a week where for some reason, for some logistical reason, forward development needed to stop for a couple of days. And so, they did on a volunteer basis they had people go after, just run RSpec profile, find the slowest test you can and try and speed it up. And they found a lot of factory [inaudible] stuff that was creating huge object trees and managed to improve their suite quite a bit. Not good enough to be useful to run locally, but still a lot better than it had been.

I think at that point then you start thinking about longer term situations and start getting into business value, culture, politics. Why do we have an app that’s this big? Should this really be three different services? Do we need to start pulling functionality out of this app and rewriting this piece by piece in services? Which is effectively what that place wound up doing. They wound up slowly killing off their monolithic Rails application by putting pieces of it into smaller services that interacted. And writing those services hopefully, although I don’t know for sure, hopefully those services will be written with faster test suites of their own and then also solve the problem of having to run the whole test suite for every change. But that is a long process that has a long-term benefit and not a short-term benefit. And I think different environments are going to be differently receptive to that.

In the immediate term, if you’re stuck in a place like that then you just do the best you can. You run the tests you can locally. You try to write tests faster, as fast as you possibly can, and start treating the existing codebase like legacy code. Oftentimes one of the reasons a codebase gets that slow is because it’s written in such a way that makes it really challenging to write fast tests. So, you do the best you can and try to make things gradually better. Trying to make things better too fast would probably cause other problems.

CORALINE:  We’re actually working on a tool right now that gathers code metrics. And one of the factors that we’re taking into consideration is the runtime of the overall suite so that that can be feedback that’s provided to development teams as they go. Hey, you guys just raised the time it takes to run CI by two minutes.

NOEL:  Yeah. I think there was a project. I want to say WebKit had a thing that if the changes to WebKit that increased the time, the CI server automatically failed it or something weird like that.

CORALINE:  Right, I remember that.

NOEL:  So yeah…

DAVID:  I temporarily found a room to talk from, but I’ll go looking here again in a minute because you may hear the echo and maybe some flushing.

[Laughter]

DAVID:  Sorry, that was…

CORALINE:  David, I feel like this is the inevitable journey of Dave Brady [inaudible].

JESSICA:  I wonder what he’s going to pick today.

DAVID:  The great thing is I’m not hashtagging this with pooping. So…

[Laughter]

NOEL:  It’s so good to know, Dave. Thanks.

DAVID:  Well, that makes it strange.

[Chuckles]

DAVID:  That makes it weird for me to be in here.

NOEL:  It’s a weird place to just hang. Is that what you’re saying?

DAVID:  Yeah, yeah.

NOEL:  Okay.

JESSICA:  But now you know how nursing mothers feel when there’s not a room reserved for them.

NOEL:  One of the things about slow test suites is that getting, pulling this, yanking this back to topic, one of the things about slow test suites is that it’s a really…

DAVID:  [Thank you]

NOEL:  It’s really a communal action. It’s a community action problem. My tests that I’m running in this small corner of it where I only care about this corner of the test suite, it’s only adding a fraction of a second or a second to the test suite and that’s almost imperceptible at the level of me working on my individual problem. It only becomes an issue over time as a whole team of developers that could potentially go much beyond you into dozens potentially on larger projects, have the test suite continue to accrue. Test suites never get smaller as functionality adds. So, your one second here, your half second here, that adds up really fast. And then suddenly you’ve got a 40-minute test suite and everybody admits that it’s a problem. And nobody’s quite sure what to do about it because everybody’s like, “Well my tests are only half a second.”

CORALINE:  Do you think it’s important then to develop a deliberate culture around testing? And I hate to use the word best practice because that shuts my brain down when I hear a cliché like that.

[Laughter]

CORALINE:  But how do you communicate and design and enforce testing practices that are not going to lead the disaster by categorical imperative?

NOEL:  Yeah, this becomes a problem. And it’s actually a problem across software complexity. It’s a problem for test suites. It’s a problem for just object structure. You don’t think a complex object-oriented structure is necessary until it’s too late. It seems like overkill until suddenly you realize you needed to do it in the past. And that happens with test suites, too. By the time the problem gets noticed it’s almost too late to do anything about it.

It’s like the old logic problem about the bugs in the jar that double in size every minute. They fill the jar in an hour. When is the jar half full? Well, the jar is half full at the 59 minute mark. The bugs start to realize that they’re about to fill the jar at the 59 minute and 30 second mark. And by then it’s too late. You need to approach this from the point of view that these are things that are probably going to feel like overkill at the beginning of a project. It’s going to feel like we’re creating too many objects. And it’s going to feel like we’re pushing things out of the database. And it’s going to feel like more complexity than we need. But if this is going to be a long-term project, then you have to start from that foundation.

And you have to start from the idea of, we’re going to try and keep this test suite under a second right now, because if it goes to even two seconds in the first two weeks, then in the first year how fast is that going to grow? And it’s really hard. I think a lot of people naturally resist that. I naturally resist that at the beginning of a project. These things feel like really big hammers to bring to bear on brand new projects. But if you start from a place, if you don’t start from a place like that it’s really hard to get to that place.

JESSICA:  So, write your tests to scale.

NOEL:  Well, I think write your tests with an understanding that they’re not going to be the only tests that ever exist. I said there are a lot of ways to be successful with this. But one of the things about all of the agile XP things, testing and pairing and stuff like that, is they all have failure modes when you only do them halfway and that they are much more successful when you do them, you either almost never do them or you almost always do them. And if you kind of do them some of the time, then you get in trouble. And that’s true for pairing and it’s true for a lot of object-oriented structure. And it’s certainly true for testing. You can potentially be successful if you don’t do TDD. You should some kind of automated testing, but you could be successful with it.

If you do TDD all the time and you really build out a nice structure, you can be successful with that, too.

Where you really get in trouble is where you’re like, well, I did testing for a while but I didn’t really refactor. And now I have these slow brittle tests that I feel the need to keep up. There’s this testing uncanny valley where you feel like you need to keep up this test suite. And we need to keep the test suite going but it’s kind of fragile and it takes a really long time to run and it’s burning a lot of our energy. And worse, it’s not even giving us the confidence that we need to verify that the code’s still working when we make changes. So, it’s not giving us that verification value over time and it’s costing a lot. And that’s a halfway point that’s a really bad place to be.

CHUCK:  So, I have to ask then, because you’re heading in that direction, what kinds of things shouldn’t we be testing?

NOEL:  What kinds of things shouldn’t you test?

CHUCK:  Yeah.

NOEL:  I think there are cases where it is possible that the cost of the test is more expensive than manual. There are certainly kinds of, certain levels, structures of view tests, kinds of view tests that are really complicated to get right and they tend to be really fragile. And they may be better served by having a manual person check things out from time to time. I think stuff that’s already covered by the framework doesn’t necessarily need to be tested.

Justin Searls makes an argument that TDD against a really strong framework like Rails is completely different than TDD outside of it because you’re not using the tests to discover the domain in a Rails environment because Rails is pretty much already giving you your domain structure. I think that in that context, you need to be careful not to test the stuff that Rails is already doing for you, because that can also lead to just a lot of tests that you don’t necessarily need to write.

I’m not a huge fan of, you see things that are like the should have matchers where you have individual tests that like, this should have this attribute, this should have this relationship, and this should have this validation, just isolated without context. I don’t find those super valuable. I prefer to do things where I’m testing at the level of, this is a behavior that should exist. And if the attribute of the relationship isn’t there, then this will fail because that needs to be there. But I like to have the tests being to the extent possible, it’s not always possible, I like to have the tests be describing the behavior of the code and not the implementation.

DAVID:  So, you guys aren’t going to believe this but I toured the building looking for a conference room long enough that the meeting in my conference room let out.

CHUCK:  [Laughs]

DAVID:  So, I’m back where I started.

[Chuckles]

DAVID:  Noel, one of the best criticisms, and by best I mean one of the ones that I’ve had the hardest time refuting because I think it has some merit, against RSpec is that it’s basically the same argument against Python or against Haml where you have this describe and then a context and then a describe and then a context, and then a context and then a context. And in each one of these, you’ve got some lets and you’ve got some stuff that’s building up state, or not state but your testable object and plugging them together. And finally, down to this thing that says, it does its thing or it mutates the transmogrificator, because that’s what it’s supposed to do, right? That’s a very readable, we’ll assume that’s a very readable test fragment. But your dependencies are 18 levels deep, well no.

NOEL:  Yeah.

DAVID:  Legitimately, eight levels deep and two monitors full past the top of the screen. What kind of a test smell is that? Or is that a test smell or is that just something, a price to be paid?

NOEL:  No, I think it’s a test smell. I think that one of the hidden advantages in Python of the whitespace is that it is a gentle discouragement against long methods and complex indentation, because that’s when Python becomes hard to read and that’s when Haml becomes hard to read. I think similarly, I think RSpec does become really complicated when you have super nested stuff. And I do try to avoid that.

And I tend to in my tests be much more forgiving of duplication in the name of readability than I would be in my code, precisely because tests depend so much on being able to see the context and because I don’t have tests for the tests themselves. So, in that sense I would be more inclined to pull things out into top-level suites and not do as much deep nesting or that kind of stuff even if that meant that I was duplication some object creation or some data like that, having some things defined in multiple places. I think that is a reasonable price to pay. I have done the multiple deep nesting RSpec thing and I usually wind up regretting it for exactly that reason. And again, that’s a thing that works fine when you have the context as you’re writing the test. But it’s a real pain when you come back to it.

DAVID:  Yeah. I’ll let myself do it one level deep. Like if you’ve got an object that depends on another object and in the context you change out the dependent object and then test the parent to make sure that it can, basically when this dependent object is screwed up this way, it should behave this way, that kind of thing.

NOEL:  Yeah.

DAVID:  So, you’re only changing one object but you’re testing the parent object. And it can look a little odd. I will only let myself do that one level deep in a file.

NOEL:  Yeah, I think that makes sense. It’s valuable to be able to do it at the one level deep, but if you’re doing it more you need to question it. Sometimes you’ll see people do weirder things where they’ll have, in RSpec the test definitions, the its are just methods. So, you can actually put them inside loops. So sometimes, people are like, here are the five user types. I’m going to loop over the same test for each of them. And it feels like a good idea because you’re saving typing. And that will bite you very, very quickly.

DAVID:  Let’s talk about that, because I just had a religious conversation with somebody recently about looping in tests. Why or why not? Or how not? Or how?

NOEL:  Okay. I think in RSpec you can loop and create tests inside the loop. And my experience is that that is usually a bad idea because inevitably, what becomes hard to do is track down when the test fails. Looping inside a test inside a test is not an automatic, I can see… I think I’ve seen something where somebody says that that inherently means that there’s a problem because you should be writing them as separate pieces or something like that.

I can see a case where I would do it, where I might do it, but I would be a little bit careful about it just because the potential for… again, it’s the same potential, the potential for having a test fail and not being able to figure out why immediately is pretty high. I do a lot of things that wind up optimizing for, without putting a whole lot of effort into writing error messages for each assertion or something like that, I put some effort into having things, I do things specifically because when this test fails I’ll have a line where I know it fails. And I’ll have an error message.

DAVID:  Yeah.

NOEL:  I do a lot of things along those lines. And avoiding but not necessarily always avoiding looping in tests.

DAVID:  Yeah. In fairness, the coworker that I was talking to was more or less in agreement with me. But she had been the victim of a religious attack about never use loops, never use loops. And I’m okay with loops inside the outer describe and outside the it statement. But if you do this, you must put your loop iterator, whatever objects you’re looping over or set of attributes. Like if you’re doing first name, last name, city, state, zip, and then you do .each and then the do says required attribute, then if your test removes that from the object and then asserts that the object should not be valid, I require that you have a context in there that says context when attr is, required attribute is missing, it is not valid.

NOEL:  Yeah.

DAVID:  So that if you fail in the loop, it tells you yes it failed somewhere in this loop. But it tells you which iteration through the loop it failed on.

NOEL:  Yeah, because that string argument in RSpec is a string and it can by dynamic. And yeah, I can see that.

DAVID:  Yeah.

NOEL:  Yeah, I can see that. I tend to do a lot of things towards trying to make, I tend to in tests, biased towards making them clear rather than making them clever more so than I do in code. So, I’ll even do things like convert objects to strings before comparing them because the error message will be clearer, because it’s an error message of two strings rather than two Active Record objects or two dates or something like that.

DAVID:  Yeah.

JESSICA:  One thing I do when I’m testing is as soon as I get an error and it’s tough to read, I go change the test so that I get a good error and then I go fix it.

NOEL:  Yeah, that’s a good idea, too.

CHUCK:  Alright. Well, I think it’s about time to get to picks.

JESSICA:  Yay, the picks.

CHUCK:  David, do you want to start us off with picks?

DAVID:  Sure. And I have a hot sauce pick today, so I’m starting a stopwatch so that I will keep myself short.

[Laughter]

DAVID:  So, today’s pick is Dave’s Total Insanity Sauce. Now, the thing about Dave and Blair’s sauces is that they were two competing brands from 20 years ago and they controlled the whole market. They had all the price points and they made very hot sauces, insanely hot sauces, and just crazy hot sauces. Dave’s Total Insanity Sauce is not as hot as Dave’s Insanity Sauce, which is odd to me. I don’t understand why being totally insane is only one-third as insane as regular insane. But anyway, it’s about 30,000 Scovilles. That’s six times the heat of a jalapeno pepper, which means you can pour this on food. Not like ketchup, but maybe like tabasco. You can splash it on.

I do have a first aid tip for people that are playing with hot sauces. Some people have tweeted at me and said they really like the hot sauce ideas. Dave’s Total Insanity is going to come with first aid tip. And that is, I believe it was Sir Isaac Newton that discovered that what goes in must come out, shall we say. And that…

CHUCK:  [Laughs]

DAVID:  Some hot sauces can cause you some grief in the morning. And this is largely a function of the physical quantity of hot sauce you put on. So, if you use a diluted hot sauce, you actually, you stand more of a chance of causing yourself some gastrointestinal distress later on. Because if you just put in a toothpick full of some insanely hot stuff, your body will just disintegrate it. And next morning, it’s gone. There’s no [inaudible] left. It’s all been digested. The sauce that I’m recommending today which is Dave’s Total, not Dave’s Ultimate, Dave’s Ultimate is 300,000 Scovilles. You’ll be off by a factor of oh my gosh. But Dave’s Total Insanity Sauce has again that nice, clean flavor. It’s got a little bit of a tomato kick to it. And this is the hot sauce that if you want to be really in the hot sauce game and hurt yourself but be able to splash it on stuff and have a little bit of kick of flavor with the heat, this is the hot sauce that you should put on your eggs.

And that’s the first food pairing which are just scrambled eggs. The second food pairing is really surprising to a lot of people, but it’s fantastic. And that’s macaroni and cheese. Make up a regular batch. Put about a teaspoon of this stuff in. It will tinge it slightly red. The more you put in, the darker red it will become. And it’s just like Satan. The darker red it is, the more evil it is. And it’s absolutely delicious and fantastic and a lot of fun.

That was going to be my entire pick but I’m going to throw one more really quick one in, which is one of my coworkers. I just started working at CoverMyMeds and I’m absolutely loving it here. And one of my coworkers yesterday gave me a welcome to the company gift. He had heard the hot sauce pick so he brought me some African fatalii peppers, F-A-T-A-L-I-I. I don’t know where he got them. I don’t know how to get them. I don’t know how to find them. But they’re almost as hot as a habanero pepper. I ate some and it lit me on fire. And I just glowed with endorphins for the rest of the day.

[Laughter]

DAVID:  They’re absolutely delicious. They’re lots of fun. Thank you, Dan. I really appreciate it. And I highly recommend African fatalii peppers. They dry them out like red pepper, crushed red pepper flakes, and you can sprinkle them on food and then hurt yourself really, really bad. So, that’s my pick for today.

CHUCK:  Alright. Jessica, what are your picks?

JESSICA:  Okay. My first pick is chapter 2 of Noel’s book, ‘Rails 4 Test Prescriptions’ because it talks about RSpec and digs into what RSpec is doing behind the magic. And as someone who has used RSpec and been impressed with the syntax but completely confused by why I have to use which magic spell where, it’s a really useful chapter completely independent of Rails testing, just for RSpec.

My second pick is maybe too late for Christmas presents by the time you read this, but a great birthday present is from ThePerfumedCourt.com. And this website, I’m going to apologize, the website is absolutely horrible. But it’s a place where you can buy sample packs of perfumes for men and women. And I recommend the 1.5 milliliter spray bottles. And it’s just really fun to get 10 or 20 different perfumes and try them out on different days. Great gift idea. That’s it.

CHUCK:  Alright. Coraline, you have some picks?

CORALINE:  Sure, I have a pick. There is a growing trend in open source projects to want to include codes of conduct for people who are participating in the project. I just wanted to mention that Contributor-Covenant.org has a semantically versioned code of conduct that people can drop into their open source projects. And I think it’s really, really important that we be welcoming the new people who are joining projects and signal that we’re welcome. So, I think that it’s a really cool idea and I just wanted to throw that out there.

DAVID:  That is very cool.

NOEL:  Great.

CHUCK:  Alright. I’ve got a couple of picks. The first one, I just want to remind folks that I am pulling together a JavaScript conference. It’s going to be online. Here in the US it’ll be after work. You can go check it out at JSRemoteConf.com and you can get users groups tickets or individual tickets.

JESSICA:  Ooh and I’m going to do a really great talk at it.

CHUCK:  Yes, yes. Jessica is speaking at it. And anyway, I’m super excited about it. So, you can check it out. You can also text JSREMOTECONF to 38470 if you want more information that way. And it’ll ask you for your email if you want email updates. I’m not going to do a lot of text updates because I think that’s a little bit invasive. But it’s just an easy way if you’re in the car or not somewhere where you can hop on your computer real quick and go check it out, if you get an email about it.

My other pick, and this is just a fun thing, is they’re called ‘Honest Trailers’. And there’s a YouTube channel with them.

DAVID:  [Laughs] Yes, yes.

CHUCK:  And basically what they do is they go through and pick apart a movie as a trailer. I think my favorites still are Twilight, which was just hilarious, and then the other one that I really liked was the Lion King. But they have them for all kinds of movies. They have it for all of the X-Men movies and a bunch of other popular hit movies that came out. So, those are my picks. Noel, do you have some picks for us?

NOEL:  Yeah. The RiffTrax version of Twilight is also outstanding.

CHUCK:  Oh, I should go check that out.

NOEL:  Yeah. So yeah, I have a couple of picks. I decided I didn’t have any really strong technical picks but I did have a couple of fiction recommendations, one of which is a writer named Max Gladstone who writes fantasy, urban fantasy, very odd urban fantasy based somewhat on the idea that modern financial systems are kind of magic. And so, the magic systems in his books bear a very strong resemblance to contracts and financial instruments and things like that. He had the experience of coming back to America after teaching in China in 2008 reading articles about Lehman brothers and thinking that it sounded like necromancy, and turned that into a fantasy series.

The books are called ‘Three Parts Dead’, ‘Two Serpents Rise’ and ‘Full Fathom Five’. They’re outstanding. They’re weird and really interesting. And he actually blogs and does a couple of really weird blog posts about the Star Wars movies including an assertion that all the people we see in Star Wars are not humans but are actually some sort of alien hive creature, which he backs up with examples from the films. It’s great.

Another writer I wanted to highlight, her name is Martha Wells, and she’s also a fantasy writer. Her best known book is probably called ‘Death of the Necromancer’ and it’s kind of a fantasy, Sherlock Holmes pastiche. She writes really, really interesting unusual alien creatures and non-human creatures. And her books are a lot of fun and really interesting.

And then one final one, a comic book pick, Astro City is a comic book series that’s been around for about 20 years now intermittently. All of the issues of it are available on comiXology and I usually describe it as everything that’s cool about superhero stories and none of the things that are awful about superhero stories. They built up this crazy superhero mythology that you see little teeny pieces of and a lot of stories about people who have jobs like the phone bank for the superhero group, the people who take in help calls and refer them around. And a lot of stories about people who have superpowers but use them to do movie special effects rather than be superheroes. And then also some cool superhero stories. Astro City. The writer’s name is Kurt Busiek. And that’s my picks.

And also of course, my own stuff ‘Rails 4 Test Prescriptions’ by the time you hear this will almost be in print, very, very shortly in print and will be available as an eBook. It’s available as an eBook right now at PragProg.com. And I have a couple of other self-published eBooks that are available at NoelRappin.com, N-O-E-L-R-A-P-P-I-N dot com.

CHUCK:  Awesome.

JESSICA:  Thank you.

CHUCK:  Thanks everybody for coming.

NOEL:  Thanks

JESSICA:  David, you made it to the end.

DAVID:  Yay.

NOEL:  Yay, Dave. You survived.

DAVID:  I have 83% battery left, even. I’m very excited. I’m so glad Noel, because I did not want to bail on you.

[Chuckles]

DAVID:  I really wanted to be here for this.

NOEL:  I appreciate that, David. I wanted you to be here, too.

DAVID:  This was a lot of fun.

[Chuckles]

NOEL:  Well, it was great to be here. I appreciate it.

DAVID:  Yeah, yeah. And Coraline, thank you for coming in on short notice.

[Chuckles]

CORALINE:  Absolutely happy to.

JESSICA:  Very, very nice.

CORALINE:  Luckily I had some coffee this morning.

[Laughter]

DAVID:  Excellent, excellent.

CHUCK:  Alright, well we’ll wrap the show. We’ll catch you all next week.

[This episode is sponsored by WatchMeCode. Ruby and JavaScript go together like peanut butter and jelly. Have you been looking for regular high-quality video screencasts on building JavaScript done by someone who really understands JavaScript? Derick Bailey’s videos cover many of the topics we talk about on JavaScript Jabber and Ruby Rogues and are up on the latest tools and tricks you’ll need to write great JavaScript. He covers language fundamentals so there’s plenty for everyone. Looking over the catalogue, I got really excited and can’t wait to watch them all. Go check them out at RubyRogues.com/WatchMeCode.]

[This episode is sponsored by MadGlory. You’ve been building software for a long time and sometimes it’s get a little overwhelming. Work piles up, hiring sucks, and it’s hard to get projects out the door. Check out MadGlory. They’re a small shop with experience shipping big products. They’re smart, dedicated, will augment your team and work as hard as you do. Find them online at MadGlory.com or on Twitter at MadGlory.]

[Hosting and bandwidth provided by the Blue Box Group. Check them out at Blubox.net.]

[Bandwidth for this segment is provided by CacheFly, the world’s fastest CDN. Deliver your content fast with CacheFly. Visit CacheFly.com to learn more.]

[Would you like to join a conversation with the Rogues and their guests? Want to support the show? We have a forum that allows you to join the conversation and support the show at the same time. You can sign up at RubyRogues.com/Parley.]

x