226 JSJ Test Doubles with Justin Searls
03:15 - Justin Searls Introduction
- Test Double
- Sinon.JS 08:44 - Mocking
- Growing Object-Oriented Software, Guided by Tests by Steve Freeman and Nat Pryce
- Jim Weirich 14:45 - Starting These Concepts as a Junior Developer
- Test-driven Development 17:55 - testdouble.js vs. sinon.js
- NIH = Not Invented Here 26:39 - Duck Typing, Monkey Patching, Duck Punching32:22 - Node.js Negativity
- Design, Resources
- Martin Fowler’s Refactoring and Patterns Books
- Domain-Driven Design: Tackling Complexity in the Heart of Software by Eric Evans 42:52 - Community45:08 - The AAA Rule: Arrange, Act, Assert51:19 - Error Messages Picks
- Unemployment (Jamison)
- React Rally (Jamison)
- Julia Evans' Tweet: how to be a wizard programmer (Jamison)
- See the good in people (Aimee)
- Sinon.JS (Joe)
- How to Stay Motivated: Developing the Qualities of Success by Zig Ziglar (Chuck)
- The Harry Potter Series (Chuck)
- RetroPie (Justin)
- NEJS Conf (Justin)
CHUCK: Jamison Dance.
JAMISON: Hello, friends.
CHUCK: Joe Eames.
JOE: Hey, everybody.
CHUCK: I'm Charles Max Wood from DevChat.tv. Quick shout-out about React Remote Conf and Angular Remote Conf coming up in October and September. We also have a special guest this week and that is Justin Searls.
JUSTIN: Hi everybody.
CHUCK: Do you want to give us a quick introduction to who you are?
JUSTIN: I would rather make Jamison Dance introduce me. [Laughs]
JAMISON: Justin… I'll do a better job. Justin runs a consultancy called Test Double which is always hiring. He talks at a lot of conferences and he gives fantastic talks. I've really enjoyed the talked of his that I've seen. And he cares a lot about testing. I feel like you write and speak a lot about testing. Other things too, but a lot of it around testing, [how to be] better at testing and how to not be worse at testing which sound like they're two different things, kind of. Is that an okay intro?
JUSTIN: Thanks. That was humbling.
JAMISON: [Laughs] Like in a good way or a bad way? [Chuckles]
JUSTIN: I think that you are really good at doing intros and you're not bad at doing intros, Jamison.
JAMISON: Oh, thank you.
JUSTIN: [Inaudible] pretty good to me.
CHUCK: So, we've had you on the show before, Justin. I don't recall what exactly we talked about then but this time we're going to be talking about testing.
JAMISON: It was Jasmine.
CHUCK: Oh, was it Jasmine?
JUSTIN: I do recall.
JAMISON: Yup, talked about Jasmine. I remember that one.
CHUCK: Well, we're still talking about testing. We're talking about testdouble.js and teenytest. I read your article that explains what testdouble is and compares it to Sinon. I have to admit, I don't know if I followed all of it. So, why don't you give a brief overview of what testdouble is? And then we can start asking questions and talking about the approach that it takes and why people may, depending on their philosophy about testing, prefer one tool over the other.
JAMISON: I have [inaudible]
First, to tell you what a test double library even sets out to do is when you're testing stuff, oftentimes having everything be real is not ideal. Maybe you're trying to write an integration test that focuses just on your code but not necessarily on the services that it depends on. Or maybe you're trying to do isolated unit tests where you're testing just a particular function and you want to verify the interactions that that function has with the things that it depends on as opposed to actually making sure that the whole thing works. A lot of people who practice test-driven development want to get that so that they can get better design feedback from their tests. And so, those are the two primary motivations for why you'd want to fake out something when you're writing a test.
And Sinon, Sinon, I'm just going to keep making up a new pronunciation every time I say it.
JAMISON: That's good.
JUSTIN: I feel like it's…
JUSTIN: Swirly? I feel like it's less thoughtfully designed than the leading or dominant test double libraries in other languages. The API is confusing. It's all over the place. It mixes up jargon like stub and spy and it's not very formal about them. It mutates a bunch of stuff without really asking. It's difficult to reset global state. It has this chainable API but it's still really hard to do a lot of very common things as one-liners. And I saw all those problems and I just kind of bit my lip for three years until we got enough at Test Double project experience of people helping teams, large teams that are writing a lot of tests again and again find that they don't understand how to use test doubles or mocks well. And they don't even really understand what they're accomplishing. And a big part of that was that they found that the Sinon API was fighting against them. It wasn't really helping encourage proper use. And I…
JAMISON: Can I…
JUSTIN: Go ahead.
JAMISON: Can I ask you to do a brief… I know this is another gigantic, enormous, [inaudible] complex topic. But can you do a brief aside into the right way to mock things out versus ways that might cause you pain? Because I know that, I've heard stuff about the dangers of over-mocking and how that can make your tests really brittle and lead to some negative consequences. But what is the right to do that? Is that something you can talk briefly about?
JUSTIN: Oh man, for sure. I could also talk long-ly about it.
JAMISON: Yeah, yeah. [Inaudible]
JUSTIN: Like I'm sure I just did about mocks. Sorry for that. And this is exactly the kind of risk, right? Is I'll just talk over everybody's heads because this is something that is really complex.
JUSTIN: But to break down as simply as I can, I think it gets back to what I was saying earlier about there being two motivations for wanting to fake something out in your tests. One motivation is I want to test this but not everything can be real. Maybe some of those services are too slow or maybe I want to be able to run my tests offline. Or maybe I don't know what. Maybe it's just a pain to get this test to work and it would be a lot easier for me to write this test if I could fake out that object or this function. And…
JAMISON: So, like the test needs a database connection or something and you just want to [inaudible].
JUSTIN: Yeah. And that's sort of the classical example. And maybe people start there. But then, because it's a powerful thing to be able to just fake out reality, right? It becomes a golden hammer for a lot of teams. And so, they just sort of as a tool of convenience start mocking stuff out willy-nilly but without a really governing principle over it. And that's where I think you get into discussions of over-mocking. Now, keep in mind too that when I said that there are two motivations, we're still on the first one and it probably represents 99% of how people tend to use mocking libraries. Because they're very convenient. They solve a very immediate need.
But when you use them in a way that isn't particularly disciplined you just end up with a bunch of tests that have some amount of decreased confidence. Like I am less, as I mock more and more stuff I am less and less certain that the thing I am testing is behaving the same way in my tests under the same conditions as it would in production. And that's where people start to hate on it, right? They can't trust the tests. They don't understand the purpose of the test. They don't know what's real and what's fake. When it breaks they don't know how all of the test doubles are set up to go in and fix it. That's the first category. Does that make sense? Of test double use.
JUSTIN: And in general, I said 99% uses that category. The other 1% of use is people making all the test double libraries. They're people who care a lot about isolated test-driven development. So, if you go back and you like at for Java there is Mockito or jMock, Ruby had FlexMock early on or Mocha, the authors for all those were trying to facilitate isolated test-driven development. Back when we were in Oslo, I did a new talk to try to illustrate what that looks like.
But at a high level it means say I'm doing a new Express route. And it's a controller. And I want to use a test to help tell me how to break that problem down. So, if I'm doing three things and I know the controller needs to do three things, rather than just write a test and then try to solve everything, I'll think upfront, “Hey, how could I break this problem down? I wish I had maybe something that went and fetched items and then something that transformed those items into data that I can, third thing, send off to a service?” So, I'm going to create three fake things to represent those three responsibilities, write a test, wire it up, make the test pass. And now I've got three smaller problems. And that's how I actually use test doubles. And the patterns of how you use them in what features you need are a little bit different.
So, you end up with a very opinionated API that has a very clear story to tell about how it's supposed to be used. But unfortunately we don't have really great marketing from the perspective of why this is a valuable approach to building software. And it gets mucked up with the complications of people who genuinely just need mocks just to fake stuff out. And I think that's really the tension that we've been dealing with all this time.
JAMISON: Sure. So, that second approach you talked about is kind of, is that something you'd see in the book 'Growing Object-Oriented Software'?
JUSTIN: Yes, absolutely. That was the genesis of it all.
JAMISON: Okay. That makes sense. Thanks for answering my tangent question.
JUSTIN: Not a tangent at all, to be honest. I think it really cut to the heart of the issue. Some people may have heard of the book GOOS. It's been called outside-in testing, isolationist testing. Martin Fowler calls it mockist testing. I started calling it discovery testing because I figured it we just keep throwing labels at a wall maybe one will finally stick.
JAMISON: Someone's going to get famous and it's going to be the person that invents that label that sticks.
JUSTIN: Right, exactly. [Inaudible] testing.
JAMISON: And you [inaudible] made out of discovery testing.
JUSTIN: Yeah, right. That's the goal anyway. I don't know. I just talked a great length and relatively granular level of detail. Does anyone have experiences that they can share about mocking or Sinon that might be pertinent?
JOE: It's awesome?
CHUCK: [Laughs] Yeah. I mean, when I was learning how to write tests, granted this was in Ruby, but I sat down. The first person I asked about mocking and stubbing was Jim Weirich who wrote FlexMock in Ruby. And as I came to understand it basically yeah, I would use mocks to isolate or basically to make it so that I don't have to worry about the particular parts of whatever I was testing that weren't core to what I wanted the test to be about. And yeah, I found that it simplified a lot of things. Because then it was, “Okay, I don't have to care if the service is down or the API server is down. I don't have to care if the database has the right data in it.” Because I can basically tell something, “Act like the database and just give back the data I want.” And then occasionally I would also get into the places where it was like, “I need to know that this was called,” because it was just mission critical that that particular API was called. And again, it just simplified things. It made things easier to keep track of. And that's where I really like mocks.
AIMEE: I wanted to back up too, and I always like to get your advice or opinion on newer developers. Because I know, so for me when I first learned how to do testing, I really didn't' start with mocking and stubbing because I think the process of learning how to test your code when you are very, very, very new, that's kind of mind-boggling in itself first. And so, when you start adding how to do an actual proper test by mocking things, like if you're not doing just, if you need to mock out things that your actual test needs it can get overwhelming. So, at what point do you think it's good to start actually using those things?
JUSTIN: I think that it's very easy when you're new to succumb to the power of the tools that you have in front of you and try to mix in a whole lot of stuff all at once.
AIMEE: Yup, yup, exactly.
JUSTIN: And then get overwhelmed because you know how to do a lot of stuff but you definitely don't understand how it works.
JUSTIN: It very quickly, you can, the why you're doing what you're doing can slip away, right?
AIMEE: That's my favorite thing I like to bring up. Exactly.
JUSTIN: So, we don't want that, right? And I think that the best developers I know are the thoughtful ones who never do anything without knowing why they're doing what they're doing. And what I would recommend is to get, if you're new to testing start by writing really realistic tests. Tests that call through your whole system just like a real user would use it. So, if it's a browser application start with tests that automate browsers. If you're building a little module that talks RESTfully to other web services, write a client that will talk to it over HTTP and then test it that way. And that way, you're getting very valuable experience because you're learning how to structure tests in a way that does the setup, invokes the thing, verifies the behavior reliably, and hopefully consistently focusing on consistency, focusing on organization, keeping stuff relatively simple. But everything that you do in that case will be valuable because out the other end you're going to have a test that you're pretty confident actually exercises the thing like reality because you're keeping everything pretty real.
Where I'd encourage people to start looking at a library like testdouble or Sinon is probably if they start to learn test-driven development. And if you want to get into test-driven development, my biggest objective typically when I'm teaching somebody is to encourage them to focus on design. You're writing a test of a new thing that doesn't exist yet? Try to imagine what things you wish it… if these things that don't exist, if they existed they could do my job for me so much more nicely, then this test that I'm trying to actually write now would become easy. And using mocks can be a great sounding board for developing those things. And there's a whole discipline and it certainly is not something we can solve over audio and explain in great detail. But that's probably the moment where I'd encourage somebody to cut their teeth on mocking libraries. As opposed to bringing it into an existing set of integration tests and just sort of using it as convenient. Because then you end up with very inconsistent use and I found that inconsistency is typically the biggest root cause of frustration that people have with test suites.
AIMEE: That's a good answer. Good advice.
JUSTIN: So Joe, you're a big fan of Sinon. But I know that you took at least a minute to check out testdouble in advance. How would you compare the two so far?
JOE: Actually I was really impressed with what you did with testdouble. I've spend a ton of time, I'm going to have a hard time coming up with your alternate. I've always pronounced it one way. So, I'm going to, maybe I should do something completely wacky. Sinon. How about that?
JAMISON: that was my alternate.
JOE: Sinon. I'll go with Sinon. You can have Sinon.
JAMISON: Okay. I'm happy to share my alternate with you too.
And you actually point out a lot of problems with the Sinon API that I've never truly noticed before. Maybe it was, I don't want to call it Stockholm Syndrome by any means. But you use a product, if it's the only product available. The only real mocking library that was out there that you can use at all is Jasmine's spy library, right? And a few people use that and it doesn't have a lot of features to it. It's very limited. So, it's okay, but Sinon was far more feature complete so I just got very used to Sinon even though I have experience with a lot of server-side language test mocking libraries, various ones, and I've seen a bunch of different kinds of APIs. So, I'm definitely opinionated on the matter. But maybe it was the only game in town and so I got very used to it. But reading your article I've seen a lot of intelligence and thought going into, “Hey, there are ways to make this better.” Your points about how the chainable API can actually cause problems because you can very easily conflict with what you'd earlier done in the chain and then it becomes very confusing as to what's going on.
So, I was really impressed with testdouble.js and thought this is actually, and without spending significant time with testdouble.js it looks to me like a next step forward.
JUSTIN: Well, that's very kind of you. And I appreciate it. I think what I'd say is if a lot of thought went into testdouble it's probably because I first had to foist upon the world a bad mocking framework for Java and then a bad mocking framework for Ruby. And I learned a lot of lessons about what at an implementation level it might look like. But the biggest thing I learned in trying to get people on board with them is that people have patience for rough edges around production dependencies they're going to use a lot. Express has some quirks in its API. And people will suffer that because they need it. They use it.
But what I've found is that with like I said a tertiary concern like mocking, people aren't just willing to devote the head space to understand really what's going on beneath the covers much less deal with the nuances presented by in the instance of Sinon a really, really gigantic surface area of its API. The number of different permutations and ways that you can do things are very, like you said, I think problematic. And for most users they don't actually experience that because they only interface with it at a very thin level. And it actually does have a lot of power. You can do a lot of things with it. But it makes it hard to find those things, especially things that you might need to do frequently.
JAMISON: Right. I want to give input as a…
JUSTIN: That's the goal. I was [trying to] popularize these ideas with testdouble.
JAMISON: I want to give input as a novice user of Sinon. I have been overwhelmed and felt the constant fear that I'm doing something wrong because I'm using four functions out of the 8,000 that it had or something. And I'm not super well-versed in mocking or the theory of it. So, I'm kind of counting on the library to guide me. And I just don't know. I'm like, I must be doing something wrong because I'm not using very much of it.
JOE: Right, yeah. Like I feel like I'm not a very good target audience to judge because even though I worked a ton with Sinon and I have a lot of exposure and experience with testing, I really don't represent the majority of people who are going to use a mocking library which is people that like you said, it's a very tertiary concern. I need to get this thing done. In order to get this done I've got to use this technology and I don't want to spend a month learning it. For me I was more than happy to dive into Sinon and learn every nuance and little bit. And I was able to compare it to something I already knew somewhere else. Whereas for somebody who just wants to quickly figure out how to do basic tasks and move forward, it's a very big, a very complex product.
Now, that being said I also felt like it was a great product. And I still feel like it's a great product. It's very feature complete. The documentation is pretty reasonable, especially for a project that's mostly handled by one guy. So, it's a great product but I like that you're taking another iteration on the whole idea and saying, “Alright, well let's approach this.” And I also feel like again, you have so much experience it's easy to get lost in the details and in the forest and create something that's easy for somebody who is highly experienced and an expert in the topic. Instead it looks to me like you've created something that's going to be easy for people to onboard with. And my first experience with a mocking library if it's this thing, if it's testdouble then it's actually going to be smoother than it could be.
JAMISON: So, can I bring up a thing I've encountered. For me, the choice whether to use Sinon or not was like, is it easier for me to just write a stub to [a] function myself and set some value inside the closure it creates. Can you talk about when you decide to use a library versus when you just change things yourself and then change them back? Does that question even make sense?
JAMISON: It just means good, yeah.
JUSTIN: Yeah, yeah.
JUSTIN: Yeah, it's just a humble brag.
JAMISON: Modular. Say modular.
JUSTIN: Super modular. Because testdouble is easy to slot in, easy to use. And if you call reset once you're confident you're not going to accidentally muck with any global state. I would use it all the time but I think that where a lot of people who roll their own mocks like that strategy is that, like we said earlier, they can definitely understand everything. And if they're really consistent about how they write and structure all their tests, they probably won't end up writing some gigantic massive regime of test helpers and fake mock stuff. Because I'm all about local fakes where that helps add clarity to the tests. I just feel like I'm always pushing against NIH especially when we're talking about testing. And for listeners who don't know NIH it stands for Not Invented Here. People reinventing wheels.
AIMEE: Hey Joe, can you actually define those for people really quick in case they don't know?
JAMISON: I know that's a thing that happens a lot in Ruby. So, maybe Justin can give us an idea of that.
AIMEE: Yeah, it was definitely for me the first time I heard those words was when I was doing Ruby.
JOE: Right. You could actually modify existing functions which is kind of an aspect of monkey patching.
CHUCK: In the middle of a running program.
JOE: Right, exactly. And then also…
JAMISON: Just like monkeys do. Hence the name monkey.
JOE: Right. In the middle of running programs, that's what they do.
JUSTIN: That the first time that I discussed monkey-patching with somebody who was a Node.js developer they used the term duck punch. And I kind of, that one's grown on me.
JAMISON: I think that's a Python thing, right?
JOE: Yeah. I heard Paul Irish use duck punching in a talk that he gave. But you know, monkey patching also kind of has a grown-up cousin to it which we call polyfills and shims. Basically kind of the same thing.
JAMISON: That's true. Yeah, I haven't thought of that. Monkey patching is what you call it when you want to make fun of it. And polyfill…
JAMISON: Is what you call it when it makes your program work.
JOE: Right. [Laughs]
CHUCK: Well, yeah I've seen monkey patches that make programs work. But the problem with monkey patches is because you've done it at runtime in the middle of the running program and you've made these changes, you may make changes that don't then meet the assumptions that other people or other parts of the program have for that particular object or function.
CHUCK: And so then when they call it and they pass in something that they assume should be an integer for example and you've monkey patched it so it accepts a string, and it can't be properly coerced, then you got a problem. Or…
JAMISON: But if it can be properly coerced then you've polyfilled it. And you've done [inaudible]
JUSTIN: Yeah, Jamison's got the right marketing mind here. You know, [inaudible].
CHUCK: Yes. Yes, he does. But…
JAMISON: I'll [inaudible] made out of polyfills.
JUSTIN: Yeah. When I want to criticize I call it a mocking framework. And when I promote my thing I call them test doubles.
JUSTIN: Because mock is such a negative word.
CHUCK: Yeah, well and I think you also illustrated well the reason why mocking makes thing simpler, is because then you don't have to have a fully inflated object with valid state in it for all of the things that I can possibly do. It just has to respond to one message and send one answer back. And so, you just create an object that duck types that one interface and that's it.
JAMISON: I think you are probably the most negative person that still uses it [laughs] that I see consistently.
JAMISON: I don't know that it's too negative, though. I think sometimes people who adopt a technology adopt it because they like it. And then there can be this tendency to feel like you need to support it. Like you have to be on the technology's side and you can't point out what it's bad at, maybe. Or you need to kind of be a cheerleader to make sure it succeeds. And I think it's really valuable sometimes to get an outside, I don't know, not an outsider because you do a ton of Node but I would say you have a perspective from lots of other communities as well where you can look at some of the things in the Node community and say like, “Hey, in community X this specific thing is way better and why can't we make it better in Node?” Was that enough of a waffle, wishy-washy non-answer to avoid getting in trouble? [Laughs]
JAMISON: Are you the waterfall guy there? [Laughs]
JOE: Hey, can I break in really quick and just say if there is a nine-week Haskell program, please let me know. I'm interested.
AIMEE: Yeah, I want to go, too.
JUSTIN: You now actually, I think a Haskell summer camp would have a pretty big market. I think there are a lot of people who'd be interested in that. But I don't think that they'd be promising 90% of candidates getting as hundred thousand dollar [a year]…
JUSTIN: Jobs in nine weeks…
JUSTIN: Programming Haskell.
CHUCK: Well, and you'd have to make sure that the business model is functional. Sorry, I couldn't help it.
JOE: Ha! Ha!
JUSTIN: But Jamison, I think what I've slowly learned, I was taken aside several times and told, “Be more constructive. Provide more solutions. Don't be so negative. And don't just make people feel bad,” which I think is what I…
JOE: Don't you just hate people that say that?
JUSTIN: I don't want anyone to feel bad. If somebody loves working with Node, well I want to do… I feel, I think that I started feeling like the oxygen had been sucked out of the room. Everyone's talking about async, small things, get up and go, build something in five minutes, adopt, adopt, adopt, start little tiny stuff and then make a big mess later. And the worry that I had was people wouldn't think about all of these other conversations. Like none of it was really informed by all of the design principles I learned from extreme programming. We don't talk a lot about domain-driven design or object-oriented programming or functional programming and what those lessons could bring to these gigantic, monolithic, Express apps that I see pop up everywhere that reinvent a million wheels.
And I see a ton of low-hanging fruit for helping a lot of Node.js teams out. And that's frankly like half of Test Double's business now is we work with teams building Node.js to try to show them the cool ways to make systems more stable that we learned in other ecosystems. Because it only makes sense that a lot of it would cross-pollinate and apply. So I'm just not, I wanted to get on the record that I'm not here to be totally just a sourpuss when it comes to Node and npm.
AIMEE: So, that's…
JAMISON: Kind of the same thing, something similar you seek? Oh, go ahead, Aimee.
AIMEE: Oh, I was just going to say thank you for asking that. Because I definitely feel that, too. That's one thing I really miss about other programming language communities.
JUSTIN: Yes. That's exactly what I mean. I think that the people who have experience in other communities, now that Node.js is not going anywhere, and I think web technologies in general have just eaten the world, that means it's a big enough tent now for us to start having conversations that might cut against or balance out some of the maybe excesses of really small things or get stuff out the door and really five minutes to five days kind of mentality that I see in a lot of Node.js developers, if not all.
JAMISON: When you say really small things, do you mean the whole tiny modules movement? [Inaudible]
JUSTIN: Yeah, yeah, yeah. The tiny modules. The anti-monolith. The anti-framework sentiment. I think that we could stand to balance things out a little that by voicing for instance sometimes what you need is a framework. If you're doing a thing that's been done a thousand times, if you're building a CRUD JSON API, you probably don't need to come up with your own custom set of routing names, you know? There's probably something that could provide that solved problem, a solution for you, that you'd have the benefit of, conventions that could translate to all your other projects instead of just insisting on building everything all over again with a whole bunch of little tiny, tiny modules that are curated person by person. These are conversations that I think happen in the corners. But because they aren't of the ilk that made Node popular, they're seen as being anti the future by a lot of the advocates. And getting back to your point about teams, I feel like they're viewed as being just spoilsports.
JAMISON: So, are you saying that these abstractions maybe exist in Node and people just aren't using them? Or just that the Node community itself just doesn't put enough effort on them to build them up?
I'm not here, I'm sure that a lot of… you look at npm right and there's 300,000 modules. I'm sure that a lot of them do exactly what I need but I just don't know about them yet. And I found that scratching my own itch by articulating the thing that I really mean for an opinionated library like I've done with testdouble or with teenytest which is a test runner that we've made, our hope is that we can use those as [concretions] to continue to carry the conversation forward. And hopefully, the community as it matures becomes more and more open to having multiple perspectives on things like what makes for a good test or a good module. And I'm just hoping to see things open up.
JAMISON: The lite edition if you will.
JUSTIN: Yeah. You know, I would go back to some of the classics. A lot of people learned a lot about just the breadth of design thinking out there by looking at Martin Fowler's Refactoring books and Patterns books back in the early 2000s. Some of my personal favorites are 'Domain-Driven Design' which is just identifying a way of getting away from the spaghetti of having a file listing that contains 15 different concerns but instead articulating a world view that tells you how to model and structure your software to map to how humans actually talk about the problems that it's solving, right? That makes it more maintainable because everyone uses that language already. There's I think just a treasure trove of books that are 10 years or older that we could all benefit from. And Chuck knows a lot of them.
CHUCK: A lot of the books? You named a lot of the books that I would have recommended. We've done book clubs on a lot of them on this show or on Ruby Rogues.
JUSTIN: Unfortunately I don't think I'll ever have it in me to write a book. That's maybe the ultimate thing to be able to say is like, “Here are all my opinions and here's the book to go with it.”
JAMISON: And they're on paper. So now…
JAMISON: They hold more weight.
JUSTIN: Yes. And I'm more important because I am now a published author. I want to have written a book. I just don't have the appetite to write one.
JAMISON: Ghost writers sound like the solution for that.
JUSTIN: [Inaudible] Yeah.
JAMISON: [Inaudible] a deliberate action.
JUSTIN: Yeah, this gets at one of my favorite phenomenon in the social side of how language ecosystems rise and fall. I got into Java when I was first learning to program in the late 90s. And you could just see this Cambrian explosion of stuff getting created as people realize they could publish a JAR to a personal website. People could download the JAR. You start to see tooling like Maven come along that make it easier and easier to suck in additional JARs for dependencies in Java. And Ruby was the same thing. The first couple of years, everything needed to be invented all over again, right? Like if you needed a thing that made an HTTP request in a nice API somebody had to be the guy who made that. And so, it's a great way to get in the ground floor and publish stuff and create a lot of very core, important tools. And Node's had that phase, right? We're past that.
The next phase is a stabilization. People start to build higher-order stuff like frameworks. People start to ask questions about well, I've got this legacy three-year-old app now and it's a mess. How do I rein that in? You know, I think that the interest in Node.js about testing has increased. When you start to get into phases beyond that, you start to look at what does enterprise adoption look like? What are their concerns that they want to make sure are taken care of if they're to adopt it or if they're to maintain it? And as things start to settle down it's inevitable that you'll have a lot of the big names who are mostly, they're excited about new, shiny stuff, they'll start to leave. And naturally you'll have a lot of attrition because a lot of the problems are just solved now. I think we're going to see that in Node.
I don't know if we'll ever see that when it comes to web technologies at large because everything is constantly evolving and there's just so much behind it. But I think you're already starting to see Node stabilize to some degree just because people like me are starting to come to the table and talk about it.
JOE: So, I read this article and I really, really enjoyed it and I liked your analysis of, “Hey, this is a problem. Here's how testdouble handles it.” And there are a few points that really stuck out at me. In fact, one thing that's interesting is you said, created these convenience functions and let me know, as far as I understand, Sinon doesn't have these. Let me know if I'm wrong. And I was looking and I'm like, “I'm pretty sure Sinon can do that.” I went and looked and yeah, I think we can do that. I say this and then you responded with this very well thought out, very intelligent, obviously something you've really done a lot before saying, “It's actually not the same thing because of this, this, this, and this.” So, I can tell right off the bat that you really understand the problem domain very well. But again like I was saying, it's pretty impressive that what you're looking for is a solution that is opinionated and leads you to the right space.
So, a couple of things that testdouble does that I really like as an improvement over Sinon is one thing that you mentioned. It focuses on a specific task or set of tasks related to testing. And rather than Sinon which tries to be everything to everybody it's got its opinions and it says, “Do things this way and do them right and they're easy and straightforward to do.” And I like that. I was not a big fan of Sinon's mocks. Sinon has both spies, stubs, and mocks. And I think pretty much stubs is a superset of spies. So, I would always use stubs. And mocks act differently than stubs. And I didn't like the way that they worked. And you actually talked about this in your article, that mocks in Sinon violate the AAA rule.
And so for those that don't know what the AAA rule is, that's the Arrange, Act, Assert. And so, this is just generic advice when you're writing tests. A test should be in three phases, right? You should arrange it and set everything up. And you should take one action and change the state somehow. And then you should assert or check that what happened is indeed what you expect it to happen. And it's been discovered over a long time when you violate that you start making tests that are less easy to maintain. And so, the way that mocks work in Sinon is that they screw up the order. You have to assert first and then you can act afterwards. And so, I really like how testdouble allows you to do the same thing that mocking does but allows you to assert last. So, for me that was a big thing. Was it hard for you to find a way to make that whole thing work or was that pretty straightforward for you that, “Oh, I want it to be like this and this is how I can do it”?
JUSTIN: The implementation for the library is pretty simple I think. And the best way to explain it is to say to anyone listening who's made it this far who's never used a mocking library before and is probably somewhat confused, if you have a fake function there's really two things that you want to be able to do with it, for the purpose of writing a test. One, you want to make it return stuff given a set of arguments. So, if I have a fake function called bark and I want it to return woof, I need to be able to configure that somehow. And the way that you do it in testdouble is you say 'td.when' and then inside the parentheses you just call the thing like you want it to be called by the thing, by your subject that's under test. And then say, then return woof. To verify it I tried to make it as simple as possible by just making it completely symmetrical. You say 'td.verify' and you can verify that interaction took place. And that's really only appropriate if the thing that you're testing has a side-effect. And so, you can't tell by return value that it did its job. And that's it.
It's just this real simple data structure essentially of keep track of what all the test doubles are. Every time somebody stubs something, throw that in an object and remember it for later. And then every time somebody invokes it, go look through all of the stubbings in reverse order and be like, “Okay, the first one that applies, like maybe it was passed with the argument one, that matches the stubbing configuration. Return that thing.” Or call back that thing or return this promise, depending on how you've configured it. And so, what I found is that if you have an opinionated API that's definitely narrow, it makes the implementation easier because you don't have to think of so many edge cases. And because this is such a simple transactional way of looking at mocking, it's really not that complex of a library.
JOE: Yeah, so I was pretty impressed that you managed to figure that and make this library handle that the whole, verify these interactions and do it in a simple way and still maintain the Arrange, Act, Assert setup that you want to see out of tests. And whenever you can't see that, it's very frustrating. And one of the big reasons why with Sinon I never used mocks was because it violated the Arrange, Act, Assert. That and it was just kind of a more unwieldy API.
I also really like this. I think you made a good choice with the whole recording API. Like you call it, you actually call what you want it to do and tell it, “When this happens, return this,” so you're kind of recording what you want it to happen. And then you make it go and if it happens according to plan, then everything works out and if not either errors are thrown or you get that whole, “Hey, you didn't call this method with the right functions or right parameters.” So…
JUSTIN: So, just invoke the function how you want to see it. And that way, you can easily grep from your tests into your source code. You can copy and paste from the test into the production code if you need to. Save a step. It makes everything a lot easier to find instead of having some poor man's attempt at imitating what it looks like to call a function by configuration, like by having a chainable API. You just call it, right?
JOE: That was awesome. Another thing I'm super impressed with is your error messages. Again, I haven't used it so I haven't gotten to the place where it's like, “Oh, you promised these error messages but you didn't deliver. In this case I'm getting a totally unusable error message,” but at least according to the documentation the error messages that you're putting out is really, that testdouble puts out is really impressive because it's very readable. It says, this is an example, I'm going to read from the documentation for people that are just listening. If you verify that what you wanted was that the method was called with a parameter of Jane for example, and what happens is it was called with a parameter of Joe, a string, right? So, the error message says, “Error. Unsatisfied verification. Wanted called with Jane but was actually called with Joe.” It just reads exactly what your problem is and gets you right to it.
And you point out that in Sinon or in something you hand-roll you might at best get, “Hey, true is false,” or at worse just get something else completely unintelligible. And it's so frustrating to work with libraries or technologies where you have to begin memorizing, “Oh, this error message, what this really means is this thing.” And you begin memorizing what different error messages or different exception types actually mean and the likely culprits that they are. But when the error message actually points you right at the problem, that's so great. One of the reasons I love Elm for example is it's really good at doing that. So, I was really impressed with [inaudible]…
JUSTIN: You know, Chuck…
JOE: Effort you put into it.
JUSTIN: Right on, man. Yeah. Chuck mentioned earlier, Jim Weirich was one of the first people he sat down with and wrote some tests that used mocks in them. And one thing that Jim impressed upon me when I was writing [inaudible] which is a spy-style test double library for Ruby a few years ago, he sat me down and he was like the most important thing in any testing library are the messages. Because the messages should be able to tell the person everything they need to either fix the test or if they're practicing TDD take the next action without having to go and print something out or debug something or take some unrelated step. Because that all breaks flow. And so, making sure that all of the testing tools that I write have really, really great messages as well as an ability like in testdouble a lot of people get confused about mocks. You can always pass any mock to 'td.explain' and it will tell you what the current state of it is, how it's been configured, and how it's been verified. Just to give you a heads up of this is what you're currently looking at instead of waiting for something to blow up.
JOE: Right, yeah. That was another really cool feature is that whole explain function that tells you exactly what happened to it. Which when you are practicing test-driven development what you want is to get into a very simple and straightforward flow of, I make a small change to a test and now it's broken. And then I make a small change to my code and it goes, it starts passing again. And I'm slowly working myself, and I never get to a point where I have to spend an hour or a half an hour digging into some weird issue. Even five minutes is too long, digging into some weird issue like, “This isn't working. How do I make this work?” You want to avoid that as much as possible. And mocking libraries are often the place where this can happen where if you have something set up and all you just get is a failure, it just says, “Nope, didn't work,” and you don't know why because you're having a hard time seeing under the covers. And these messages, letting it know, “Hey, you called it with this parameter then you called it with these parameters and then you called this other function with these parameters,” gives you perfect introspection into what's going on in your code. And I really like that feature as well.
JUSTIN: Awesome, man. Thanks. I really appreciate it. I'm really unusual in that I very much like compliments.
CHUCK: Alright. Well, I've got 15 minutes…
JAMISON: We'll keep [inaudible].
CHUCK: Until the next podcast. So, we need to get to picks. Jamison, do you want to start us off with picks?
JAMISON: Yes. My first pick is unemployment. I got [inaudible].
AIMEE: No! So sad. [Chuckles]
JAMISON: I know.
AIMEE: But happy.
JAMISON: We parted amicably. I like Kuali. It's a great place to work if you're looking but I'm going to be taking a month or so at least to do nothing and just be a dad and do React Rally stuff. So, I'm looking forward to that. I've never really done that in my life.
Which leads into my second pick, React Rally. React Rally is a conference that Matt Zabriskie and I are putting on in Salt Lake City, August 25th and 26th. We're just finalizing preparations for right now and we're really excited about how it's shaping up. It's focused on React. Hopefully you got that from the name. But really excited about the speakers we have, the events that we have, the theme and the setup of the conference. I think it should be a really good time for attendees and speakers and everybody. So, check that out.
My last pick is not self-serving [chuckles] for a change. It's just a tweet by Julia Evans where she talks about how to be a wizard programmer. I really, really, really like her thoughts on programming and development. And this one, she talks about how important the skill of asking questions and being around people who can help answer your questions is. But then after that eventually if you keep doing that, you'll get to questions that other people can't answer. And that's where you have to figure out the answer. And you might be the first person ever to ask that question and figure out the answer. And that's what makes someone a great developer.
So, those are my picks.
CHUCK: Awesome. Aimee, what are your picks?
AIMEE: Okay. So, I actually don't have any technical picks this week because as we were talking about at the beginning of the show I've been going through a lot of just personal things that will in time probably come out. So, it's been a really, really rough couple or three weeks for me. And so, my pick this week is probably going to be to take a step back and as you look around and you're just going through your day-to-day, one thing that the pastor did a sermon once that I really, really liked is if you're on the road and somebody cuts you off and you get angry with them just maybe take a deep breath and try to think. You don't know necessarily what that person is going through. Maybe they're rushing to the emergency room because someone in the back of the car is really sick. Or who knows? But I guess that's my pick this week is to just kind of try to see the good in people. So, that's it. [Chuckles]
CHUCK: Awesome. Joe, what are your picks?
JOE: So, this is going to sound kind of funny but my first pick is going to be Sinon.JS.
JAMISON: How rude. Depending on what your second pick is.
JOE: Oh guys. Thanks for putting under some pressure. No, we've been talking a lot about some of the problems of the Sinon API and how testdouble.js seeks to solve those with a slightly more opinionated and focused framework, which I really like. But I think one thing that could be missed in all of this is the fact that Sinon solves a problem well and is a great tool. Maybe it's the day of testdouble has arrived and it will slowly become the de facto mocking library. But that doesn't take away what Christian Johansen did with Sinon. And so, I want to pay my respect to that. We had him on the show long ago, many years ago. And I was really excited to talk to him because I use Sinon a ton and was really impressed with the work that he did and the pioneering that he did to make a really good mocking framework. So, I want to pick Sinon as a show of respect and just all the difficult work that he did to give us something that really worked. And so, that's going to be my pick.
CHUCK: Alright. I've got some picks here, mostly not technical picks. I've got a whole bunch of stuff going on this week. I'm going to be out next week because I'm going up to Wood Badge which is Boy Scout leader training. But this last week I read a book, or listened to a book on Audible. It's by Zig Ziglar and it's 'How to Stay Motivated'. And it's kind of funny. I just feel like I have better ideas about how to do better with life. It's not any one grand idea that makes things better but just a whole bunch of principles that you can apply in your life that help things get better.
So, I've actually set the goal to listen to everything that Audible has from Zig Ziglar that's in English, because they have Spanish programs and stuff on there too. I'm not a fluent Spanish speaker. I get better with Spanish as it gets closer to Italian. We'll put it that way. But anyway, super great books. I'm currently reading 'Over the Top' and 'See You at the Top' which are two of his other books. They're not very long. The audio programs on Audible are terrific because he's actually speaking. He's not just reading the book. And they are terrific. I know Zig died a few years ago but what he's teaching there I think are sort of timeless principles for people to be successful in life. So, if you're looking for something like that, then definitely check it out.
And then the other pick I have, I read these books quite a long time ago. I think most people have at least heard of them. But I've been rereading or re-listening to the Harry Potter books. And it's just a really nice way to relax. So, I'll just put that out there.
Justin, what are your picks?
JUSTIN: Today my first pick is going to be an [OS] distribution for Raspberry Pi called RetroPie that kind of comes out of the box. It lets you throw a bunch of NES Nintendo game into its emulators and you plug it into a TV and then you've got a deck of card sized retro game console. It solves a real problem for me which is I travel three quarters of the year the last couple of years and I [inaudible] each night. And so, I get really, probably the thing I hate most about traveling [inaudible] a rhythm of gaming. So, I think I'll be able to travel with it. Now the downside is it's in this clear Plexiglas box. And there's memory chips and stuff poking out. So, I'm sure that the TSA is going to confiscate it before [inaudible]. But at least as long as it lasts, I'm excited to have it.
CHUCK: Alright. If people want to follow up with you, get to know you better, they want to get hired by Test Double, or they want to hire Test Double, what do they do?
JUSTIN: Easiest thing to do is to just email us. You can say firstname.lastname@example.org and say hi and we'll find the time to have a chat. You can find us obviously at TestDouble.com or there on Twitter. If talking to a corporation is too scary, I'm Searls on Twitter. And I've got open DMs and I'm always happy to talk to anyone about anything. Even if you're just looking for some free tech support or free advice.
CHUCK: Alright. Well, I know that the internet tends to misspell stuff, so how do you spell Searls?
JUSTIN: Searls, so my wife's a teacher and she teaches her kids every year on day one that you spell Searls or pronounce it like the word pearls but with an S instead of a P. So, it's S-E-A-R-L-S.
CHUCK: Alright. We'll have that in the show notes, too. But I know some people just listen to the show and then they might tap it in on their phone. So, alright. Well, we'll go ahead and wrap this up. Thanks for coming and we'll catch everyone next week.
[Bandwidth for this segment is provided by CacheFly, the world’s fastest CDN. Deliver your content fast with CacheFly. Visit CacheFly.com to learn more.]