106 JSJ Protractor with Julie Ralph

Download MP3



01:32 - Julie Ralph


Next Week

ClojureScript, Om, etc. with David Nolen


JULIE:  Everything should be a pun. [This episode is sponsored by Frontend Masters. They have a terrific lineup of live courses you can attend either online or in person. Their upcoming course is Hardcore Functional Programming in JavaScript with Brian Lonsdorf. You can also get recordings of their previous courses like JavaScript the Good Parts, AngularJS, CSS3 In-Depth, and Responsive Web Design. Get it all at FrontEndMasters.com.] [This episode is sponsored by WatchMeCode. Have you been looking for regular high-quality video screencasts on building JavaScript done by someone who really understands JavaScript? Derick Bailey’s videos cover many of the topics we talk about on JavaScript Jabber and are up on the latest tools and tricks you need to write JavaScript. He also covers language fundamentals, so there’s plenty for everybody. Looking over the catalogue, I got really excited and I can’t wait to watch them all. Go check them out at JavaScriptJabber.com/WatchMeCode.] [This episode is sponsored by Component One, makers of Wijmo. If you need stunning UI elements or awesome graphs and charts, then go to Wijmo.com and check them out.] CHUCK:  Hey everybody and welcome to episode 106 of the JavaScript Jabber Show. This week on our panel, we have Joe Eames. JOE:  Hey there. CHUCK:  AJ O’Neal. AJ:  Yo, yo, yo, coming at you live from Provo, Utah. CHUCK:  Aaron Frost. AARON:  Hello. CHUCK:  Jamison Dance. JAMISON:  Hey friends. CHUCK:  I’m Charles Max Wood from DevChat.TV. And we have a special guest this week, and that is Julie Ralph. JULIE:  Hey guys. CHUCK:  Do you want to introduce yourself for those who don’t know who you are? JULIE:  Sure. So, I’m a software engineer in test. I work at Google. And I’ve been working with the Angular team doing testing work for the past year, year and a half or so. And I’m the primary author of Protractor, which is the end-to-end testing framework for Angular. CHUCK:  Very cool. JOE:  Nice. CHUCK:  Well, we brought you on today to talk about none other than Protractor. AJ:  So, does that mean that Igor and Miško, they’re your assistants? JULIE:  Precisely. [Chuckles] JULIE:  No, Igor tells everyone what to do. And so, he tells me what to do. [Chuckles] JAMISON:  So, I know sometimes there’s an adversarial relationship between QA and development. Are you the sharks and the jets and [inaudible]? JULIE:  [Chuckles] So, I think that Google has a very healthy relationship between software engineers in test and software engineers in general. And the basic relationship that we try to cultivate is that engineers should do their own testing in general. So, software engineers in test aren’t there to write your unit tests for you or be the people who bother to write unit tests. The goal of a software engineer in test is really to be someone who’s developing tools that make it easier to write tests or that enhance productivity. So, we often call ourselves engineering productivity as an organization as well. So, we’re just devs who work on testing frameworks. And we write our own tests and everybody else should write their own tests. JOE:  Cool. I love it. AARON:  So, you’re a software engineer in test at Google. JULIE:  Yes. AARON:  So, what office are you located out of? Are you in Mountain View? JULIE:  I work out of beautiful, sunny Seattle. AARON:  Cool JULIE:  Which is actually beautiful and sunny today. JAMISON:  Wow. JOE:  Did you say sunny and Seattle in the same sentence? JULIE:  I might run out halfway through this podcast to go enjoy it. [Laughter] CHUCK:  they only get one more day like that a year, so. AARON:  Yeah. JULIE:  Yeah. It’s precious. JOE:  Are you talking about Seattle, Arizona? [Laughter] JULIE:  Is there a Seattle, Arizona? JOE:  Is there a little town in Arizona named Seattle? CHUCK:  Now somebody’s going to go google that, testing google. JULIE:  There probably is. JOE:  Yeah [laughs]. JAMISON:  Do you want to tell us a little bit about what Protractor is? And maybe it’d be helpful for me to know how it differs from things like Karma or some of the other testing things out there for Angular. JULIE:  Yeah, sure. So, I think the best background is probably a couple of years ago the status quo for doing end-to-end testing for Angular was the Angular Scenario Runner, which was a runner that loads up your application inside an iframe in the page and does end-to-end-ish tests through a purely JavaScript page talking to page and iframe API. And this was nice because it was really easy and you didn’t need anything special to run it. And its API was very, very readable and very nice syntax. But it had a lot of limitations. So, the way that I got into Angular was I was actually working with a project that was using Angular at Google. And we tried using the Scenario Runner. And we found that it really just couldn’t do the things that we needed to test. We needed to test things like logging in. And your login page isn’t an Angular page. And the Scenario Runner couldn’t deal with that. And it couldn’t deal with things like popups and multiple windows, navigating the browser history, stuff like that. And the status quo for testing in general at Google is to use the WebDriver project which is a standard for testing and interacting with applications like a real user would interact with them. So, the way that WebDriver works is it has an API and there are various drivers for various browsers which implement that. And they say that they agree to the specification on. They’re going to add native real events that act like real people. And so, Protractor uses WebDriver to test Angular sites. And what it does is it uses the fact that your site is in Angular. And so, we know all this additional detail about a site that we wouldn’t know about just a generic webpage and makes using WebDriver easier. Gives it automatic synchronization with your page so you don’t get race conditions, gives you additional locators for finding elements that you’ve already added something like ng-repeat or ng-binding to so that you don’t have to use complicated CSS to get at that. And it’s also just bringing WebDriver to something that’s more accessible that you can easily download and run from the command line using node, writing your tests in JavaScript which is probably the same language that you’re spending most of your day programming in. CHUCK:  So, it seems like, I was reading through the docs and it mentioned Selenium or WebDriver and it mentioned Jasmine. So, is this just a loose framework that's bolted together that uses all these tools and then make the assumptions like you said, that it’s an Angular app? JULIE:  Yeah, that’s basically it. It’s bringing together a familiar testing framework. And I realize I didn’t really mention the relationship with Karma, which is that Karma is much more designed for unit tests, tests that can be run really quickly, tests that can be run on save but doesn’t deal with things like native user events or interacting with the page as a user really would. And so, I really encourage using Karma. Get your unit tests set up. Use Karma for that. And then use Protractor for your end-to-end tests. But Protractor uses projects like Jasmine for its framework so that it can look similar to your unit tests, so that you’re not in a big context switch when you’re writing a different type of test. JOE:  So, what would you say about shops that decide to use just one tool or the other and not both? JULIE:  [Laughs] I would say not a great idea in general. Honestly if I had to choose, I would say go with a really strong suite of unit tests. Unit tests are your first line of defense for finding bugs in your project. I don’t see how anyone could make anything that they could maintain in any way without a unit test suite. That said, I think there’s a lot of value to having at least some end-to-end tests. On every project that I interact with, I get the end-to-end tests setup. And then at some point that will come, it will be like, “Hey the end-to-end tests are broken. Can you help me debug it?” And we’ll go and be like, “Okay, can we start up the demo server?” And they start up the demo server and something’s broken with the demo server. It’s not the tests. It’s the tests have actually bug and there’s something that doesn’t connect right and the dev didn’t think that their little tweak would break authentication. And so, there are just so many things that can go wrong when you actually put an entire application together that having just a simple end-to-end test that pokes a couple of buttons can give you a lot of value and a lot of protection against those kind of issues. JOE:  That’s interesting. I was hoping you’d use more words like ‘idiot’. [Laughter] JOE:  But your answer works, too. AARON:  That was less inflammatory than what Joe wanted. JOE:  Yes. JULIE:  Yeah. I don’t find that getting people to write tests through yelling works particularly well. JOE:  Managers have tried it for ages. “You’re not doing it right.” [Laughter] AARON:  YDD. [Laughter] CHUCK:  Yelling-driven development. Is that… JULIE:  You should write a book. AARON:  Yeah. JAMISON:  [Chuckles] JOE:  Yeah, totally. JAMISON:  You’re now thought leader on YDD. AARON:  Yup. JOE:  So, I think that I’ve been in a lot of discussions about test counts and test runtimes and stuff like that between end-to-end tests versus unit tests. I’m curious to hear your perspective on that being as that you are the only engineer in test that I know very well. JULIE:  Yeah, so end-to-end tests are going to take longer in general for test runtime and this is something that I don’t like. And I especially don’t like that when you have to get your environment hermetically set up and maybe your browser that’s actually running your tests is in a server farm somewhere because you need to run your tests on an IE Windows machine and you don’t have one in your project on the desk in front of you. And so, then the talking back and forth between that machine takes a whole lot of time. So, tests can be super slow. The Angular project itself’s test suite takes about 30 minutes now. And we’re really not happy with that. That’s way too long for end-to-end tests. So, that’s something that we would like to work on. In terms of where the balance is, I would say whenever you change the code you should probably be adding a test for it. If it’s something that you can test with a unit test, that’s great. If you found a bug with something that is better tested with an end-to-end test add an end-to-end test. But really, try to lean towards having more unit tests. AARON:  So, you said that your guys’ test runtime is 30 minutes and that’s too long. I come from a QA background. I’m wondering Julie, what do you do to cut down to 30 minutes? What are some of the things you can do? JULIE:  So, the quick answer is to parallelize things which we’ve been working on. It used to take an hour and 30 minutes until we split it up into running the jQuery tests on different browsers versus the jQLite tests on different browsers so that everything can actually run at the same time. And we use tools like Travis and Sauce Labs which let you do that, which is great. So, parallelized tests, and we’re working on changes to Protractor that make it easier to even parallelize running your tests in the same browser, in the same browser type, so in Chrome but at the same time. So, it takes the same number of resources. And you always have to consider the startup cost for that. But it can give you a lot of gains overall. Not writing silly tests is another thing. It’s hard to get your mind in the state for end-to-end tests, because often for unit tests the setup time isn’t really important. So, you really do want to clean everything up and start from scratch at the beginning of every test. And that might not make the most sense for every end-to-end test. Maybe it can be one big test case that handles a couple of different things. And that way, you don’t have to do a full grab of the page before testing those two or three different things. They’re really part of one user flow, so they can be part of one test case. AARON:  That’s what I was hoping you would talk about, because I think a lot of people, if they take the same approach they take towards unit tests then every time they want to do something they have to create a user and set up some sort of piece in the system. And they have to do all the setup and all the teardown. But you’re saying… Where’s the limit though? When is your test too big or what are some guidelines you follow when you’ve crammed too much into one test, is what I’m trying to figure out. JULIE:  Yeah. It’s definitely a balancing act. And I think if your test becomes unreadable, and that’s a little bit of a fuzzy definition, if someone’s finding it hard to understand or if you yourself are ever finding it hard to understand, it’s probably too big. And if it’s ever outside of the scope of something that a real user would actually do, then I think it’s also getting a little too big. JAMISON:  So, it sounds like you’re saying you should limit your integration, or the more expensive tests to the happy path to your app, not use them to test corner cases or weird things. Is that what you’re saying, kind of? JULIE:  I think there are a couple of types of weird things. If it’s a weird thing that you can catch in unit tests like validation for a form, you don’t want to check every type of form validation using and end-to-end test because you could just take forever on that. But other things like what if you get an error from the server? That’s something that you might not be able to test really well in a unit test where there’s no real server and no full UI and you want to see. If you have a butter bar, does the butter bar show up? So, I think you have to pick carefully the bad cases that you want to test and things that are related to what’s going on with the server, what’s going on with something like user authentication or interaction with third parties, are probably the areas where you’ll get a lot of benefit. AARON:  I like that. CHUCK:  Now, one thing that with my experience, I’ve used Selenium WebDriver before with tests. And that’s where my tests tend to get long. So, are there particular things that you can do with those tests to get them to pare down a little bit and not be so long-running? JULIE:  All Protractor tests are using Selenium WebDriver so you’re talking about just WebDriver communication with the browser is what’s taking a long time? CHUCK:  I don’t know if that’s why they’re slow or if there are other things. But you generally have to, for example wait for the page to load and for the JavaScript to run, which sometimes can slow it down. But if you have a long-running test suite and you’ve got that kind of integration in there, then that’s the kind of thing that slows it down that I’ve seen. The unit tests are usually pretty darn fast. JULIE:  Yeah. And there are some things that you can do. So, for Angular’s own tests, we do things like we turn off animation before the test is even loaded. So, Protractor has this neat little feature where you can go in and mock out or modify any modules before Angular actually instantiates its injector and loads up all of its modules. And so, you could go into the animate module and turn that off for your tests. And that gives us 40% of time improvement for a lot of Angular’s own test suite just because it’s not waiting for things like that. So, you might be able to go in and do some trickery to make your page load a little faster. AARON:  So, you guys are making it easier to parallelize stuff on the same machine, it sounds like, so you won’t have to farm out a bunch of different Protractor servers. Can you talk a little bit about that? JULIE:  Sure. So, there have been a couple of requests that have been hanging around for, I want to be able to shard my tests, I want to be able to run different tests in parallel. And I’ve been thinking for a while about how to do this without just causing a nightmare of debugging and trying to figure out how the launcher decided to shard your tests and why some of them fail in odd ways that are unpredictable. And flaky tests are better than no test at all, I always like to say. And so, I think I’m coming to some conclusions about maybe a test file is a chunk that should never be split up. And that should just be well documented and printed. And Protractor can go in and split up you files between different instances of the same browser type and run those. And you’ll still get some issues where if you’re hitting the same webpage, you could get race conditions. So, I think a lot of the trickery here is also around just adding documentation about, hey if you’re keeping a counter that’s global for all users and you have multiple tests hitting it at once you have to be careful about that. And that’s something you have to be careful about in real life, too. So, these are real situations and real user scenarios that you’re testing. CHUCK:  Is there a way to tie this into CI systems like Travis or Jenkins? JULIE:  Yeah, definitely. So, Protractor itself you can go see on GitHub has Travis set up. And it uses Travis to host a little test application and actually goes and runs the test on Sauce Labs. If you don’t know what Sauce Labs is, they’re a Selenium farm basically. So, they’ll give you all sorts of different browsers to run against WebDriver. Really easy setup. They’re awesome. We use them for Protractor’s own test suite and for Angular’s own tests. And then if you need to go back and debug, they’ve got a video of what happened for your test. They’ve got the whole logs and exactly which commands were sent and what the responses were. So, Sauce Labs is super great. And you can also use them to increase your computing capacity if you need more browsers, or as I mentioned need browsers that you don’t have on your local machine. We do use Jenkins as one of the CI servers that Angular, the Angular.js project has. And they also use Protractor to just run locally on the Jenkins machine. CHUCK:  Yeah, it makes sense. So, every time somebody checks in what gets run versus maybe… do you run the whole suite or do you just run certain parts of it that are really fast? JULIE:  Angular runs the whole suite. CHUCK:  Okay. JULIE:  Protractor runs a subset of tests itself actually, because a lot of them are about failure cases. And maybe there would be a better way to run those on the continuous integration server, but at the moment they require a little bit more just manual checking in and making sure that things failed in the right way. And there’s no meta-framework at the moment to test that. But yeah, the whole suite for Angular is run. AARON:  Yeah, I had a question. I was going to ask about, so you guys just came up with the idea of Protractor. What was it? You said that the spec runner wasn’t good enough. So, did you just sit down one day and bust this out? Or, who was the team that worked on this with you? Or, where did Protractor come from? What was the root? JULIE:  So, it was mostly me. So, I mentioned I’ve been working on this project that was using Angular. And we were getting annoyed at the Angular Scenario Runner and wanted to do something else. And I went and talked with the Angular team a couple of times. And they basically said, “Yeah, the Scenario Runner is in maintenance mode. We know it’s not perfect. If you want to do something with this, go ahead.” And then the team that I was working with moved to New York. And I found myself… [Chuckles] JULIE:  In search of something to do. I didn’t want to move to New York. So, I have a very chill boss who was happy to let me work on this. There were other teams internally that were interested in a better end-to-end solution for Angular. I went to talk to the team again and they say, “Yeah. But you got to make this work for everyone outside, too.” So, that was how it started. In the same way, being an open source project, GitHub first. And then bringing all that goodness back into the internal Google system. AARON:  Cool. You hand selected the tools that make up Protractor? JULIE:  A lot of them were selected because they have a lot of strong support and infrastructure already around them in big companies, like what I see internally at Google. We have great infrastructure for provisioning browsers already that use Protractor. So, a lot of it comes from what we had already built for non-Angular applications. AARON:  Cool. JULIE:  And then the other part comes from lessons learned from the Scenario Runner and lessons learned from Karma. So, if you use Karma and Protractor, you could see there’s a lot of similarities in terms of what the configuration file looks like and how the command line tool behaves and what the syntax of the tests look like. And you can use the same Jasmine syntax. You could switch to Mocha if you want to and build frameworks. And a lot of the syntax, the microsyntax of Protractor itself, which if you look around will be things like browser and element being the globals that do the main interaction with the page and the main interaction with grabbing elements, those are pretty much lifted from the syntax that the Scenario Runner had come up with. AARON:  Cool. Cool, cool. It sounds like you put a lot of heart into it. And there’s a story that I’ve been interested to hear maybe your point of view on. I heard Brad’s take on it. And maybe he was being funny, but at ng-conf he talked about you walking into his team and saying, “I want to be on your team. Make me a member of your team.” And they were like, “Oh, geez. Okay. Let’s make her a member of the team.” So, how did that play out? Did you really pound the desk and make [inaudible]? JULIE:  [Chuckles] I don’t think that actually happened. AARON:  Okay. JULIE:  [Laughs] No. I think that the real story is probably somewhat less exciting in that I was on a project that was using Angular in the more early days before stuff really exploded. And I got interested and really liked working with it and was really attracted to how testing was baked in to Angular and its dependency injection system and its great unit testing story. And so, it was really a pleasure to work with from that point of view. And so, I wanted to be more involved with it. It is important to note that Brad is not my manager. So, we have a slightly complicated structure around where people report to. And there are a lot of dotted lines. So, I like to consider myself a far overblown 20% contributor to Angular, where it’s really more like 90% of what I do. [Chuckles] AARON:  That’s funny. And your boss is cool with you working with Brad’s team? JULIE:  Yeah, he’s great. AARON:  That’s cool. CHUCK:  Thank you, Julie’s boss. AARON:  Thank you, Julie’s boss. What’s your boss’s name? JULIE:  Arif. AARON:  Arif. Thank you, Arif. JAMISON:  I want to ask a wildly different question. It goes back to something you mentioned at the beginning of the episode where you said that everyone in Google writes tests. You don’t have a separate QA department. Am I saying this correctly or am I misremembering? JULIE:  No, no. That’s right. JAMISON:  But you still have separate teams of engineers who are dedicated to making it easier for the normal developers to write tests? Is that correct? JULIE:  Yeah, that’s right. JAMISON:  So, there’s a wide spectrum of opinions and practices for QA. Some people have manual QA departments. Some people have no QA at all, like what it sounds like Google has but you just trust the developers to make stuff that works and have automated tests around it. How do you balance, or how do you decide what the best practice is? Does that question make sense? JULIE:  Yeah, I think so. And I would like to add a link to the Google Testing Blog, which I’ll put in the show notes. I think that has some really great posts from people in various corners of Google who are software engineers in test talking about what they do and their relationship with the teams that they work with. I think you’ll find, I’m sure people have had this experience, that when you write your own tests, your code is a lot better than if you’re trying to just go to a spec that someone else has given you or trusting that someone else will write the tests or not writing tests at all. And I think that’s the philosophy that Google goes off of. We also just don’t have enough software engineers in test to make us writing all the tests be viable. I think depending on the product area, a 10 to 1 ratio of plain old software engineers to software engineers in test. So, it’s just not feasible to do it. So, we need to engage with teams in other ways. And we’ve had projects in the past, things like test [certibility]. So, your team could become test-certified and go through levels of little check boxes for, do you have unit tests, do you have integration tests, have you done a performance test, and safety stuff And the role that the software engineer in test was more to guide people through getting to those stages and showing them the best practices and the current status quo for tools. So, I think the role ends up being a lot more writing tests, helping out with the release stuff, writing infrastructure, writing example tests for the team. And I think that it’s a culture thing in a lot of ways. You have to convince devs of the value of writing tests and make it the normal. And make it so that when you do a code review before anything is checked in, if it doesn’t have tests, it just immediately gets sent back. JAMISON:  That’s really interesting what you said about writing the example tests because I feel like lots of the work can be in blazing the trail in some cases, especially if you have a large established codebase that isn’t very well tested. That initial effort to get the first test is just so large that it can feel hard to set aside enough time to work on that. JULIE:  Yeah. JAMISON:  That’s a really interesting idea. JULIE:  That’s something that can be a little bit tricky with Angular as well, just in terms of unit testing. Like I said, I think Angular has a great story around unit testing and I love that about it. But it can be a little bit different. And so, I’ve seen a lot of teams that just need an example of how do I set up a unit test for a directive? How do I get the modules that I need for that? Do I attach it to the DOM or do I not attach it to the DOM? And so, we just have a little set of a couple of example tests. This is a test for a really simple directive. This is a test for a really simple service. This is a really simple service where you mock out something before you do the test. And those are all the unit tests. And I think there’s a lot of value to just having examples to go off of. JAMISON:  I’m stroking my mustache in thought right now. [Chuckles] JAMISON:  Interesting. AARON:  Does Angular 2 completely break Protractor? JULIE:  At the moment, yes. But it won’t. [Chuckles] AARON:  Oh, good. JULIE:  I’m working with the team. And Angular 2, you could go see the design docs and see how everything is shaping up. And testing remains important. So, testing’s going to be important. We’re probably going to have to have a separate branch of Protractor to test against Angular 2 because right now it does rely on a lot of APIs that are very specific to the way that Angular 1.x works right now. The other thing that I’m working on with the Angular team is even in the 1.x, so probably in the 1.3 branch, helping unite all of these APIs that are sometimes private things that Protractor hooks into, into something like a testability service. So, that would make the API a lot cleaner, push some of the work from Protractor side into Angular side. But it would be a nice interface that not only Protractor could use but might make it easier to debug if you wanted to work with that, or projects like Batarang which is a Chrome extension that helps do debuggability and add that. And that could use the testability API as well. AARON:  Great. Well, I’m excited to see how that one works out. I know that you probably hear that more than you’d like. But we’re excited to see what Angular 2.0’s got in it. [Chuckles] JAMISON:  But no pressure. AARON:  Yeah, no pressure. [Laughter] CHUCK:  If you want to know, go bug Merrick. He was working on it at the hack night the other night. AARON:  Yeah. JULIE:  If you want to know, there are design docs. Anyone can go read them. AARON:  Yeah. I read some of them. And some of it I’m excited about. Some of it I’m nervous about. But that’s why I want to see it. So, we’ll see. CHUCK:  Very nice. I’m super excited to see how it all works out. And just talking to Merrick and getting an idea of what’s coming up in Angular 2 is, I’m just thrilled. I think it’s going to be really, really cool. JOE:  So Julie, I know that you talked a little bit about testing at Google. But I’m curious. The Angular team specifically, I think that one of the reasons that Angular has become so popular was its how, important testing is with Angular itself. Karma and now Protractor are basically spawned in some way or another around the Angular project, right? And so, as history looks back, we’ll definitely credit Angular as bringing Karma and Protractor about. Do you feel like the Angular team is even more Nazi about testing than Google as a whole? JULIE:  [Chuckles] Yeah. I wouldn’t put it that way, but that is definitely true. And Miško was very involved in projects like internal testability, even before Angular. So, the Angular team overall is extremely strong on testing. And they’re the kind of guys who don’t require any external pushes to do the tests right. They all understand that if you can’t test something, you don’t have confidence in it and do a really great job of making sure that everything works. JOE:  Before we get into something potentially controversial coming up, I want to ask another question. [Laughter] JOE:  I played with a really early version of Protractor. And I was using a Windows box at the time. And one of the things that really stuck out to me was that there were a lot of moving parts to get Protractor up and running. I haven’t played with it recently. Can you speak to that? And especially speak it to running Protractor on Windows. JULIE:  Absolutely. So, there are a lot of running parts. And we’ve really been working to reduce how many you see. There’s going to be some running parts because this is based on Selenium/WebDriver as I mentioned. And that means that you’re going to need the WebDriver server, something that’s driving your browser, and then the browser itself on your machine. So, you’ve already got a couple of parts moving on there. In more recent versions there’s a binary that comes with Protractor that’s called the WebDriver Manager. And if you install globally, that just gets added to the command line. And it’ll automatically download, update, and start up WebDriver for you. So, that should help out with that piece. And I think we also need better documentation around things like using Sauce Labs if you don’t want to have your own locally hosted browser. The other moving piece is that you need an application to test against. And unfortunately, there’s not really much that Protractor can do to help you get your own local server set up. But that’s a big problem. And it’s a big problem deciding, are you testing against a staging environment that’s just being hosted somewhere and has a persistent environment? Or do you actually want to bring up a whole new dev environment every time you run your tests? And there are pros and cons to each. Obviously, the time that it takes to bring up a dev environment is a big one. But then you could maybe inject all of your own data to that one. So, you would know exactly what was going on in that server instance. So, there are pros and cons. And unfortunately, you need to think about that for your own project and decide what the best way to do it is. JOE:  Well, it really seems also though, if you have an overly complex application to launch and to get up and set up and run, and you have to have it running in order to use Protractor, then it’s going to be very painful to use Protractor. And some people might look at that as a drawback. But me personally I would look at that as an advantage, because that’s going to make you feel the pain now and make you want to fix that pain by simplifying how your application launches and runs and maybe reducing the number of moving parts in your application so that it makes it easier to work with Protractor. JULIE:  That is definitely an advantage of end-to-end testing in general, is it really makes you face the complexities in your application. JOE:  It’s a lot like continuous deployment. If it’s painful to deploy, then it’s painful even more so to put it in continuous deployment. JULIE:  Exactly. JAMISON:  So, I want to stir the pot a little bit and be very topical. Today is the opening day of Rails Conf and DHH gave a keynote. CHUCK:  Oh, he’s going to stir my pot. That’s what he’s going to do. JAMISON:  Yeah. Well, I think it applies to [inaudible]. And one of the things he talked about is how developers are sort of fashion obsessed in a way, where there are lots of fads in technology. And he called TDD the most successful fad diet of all time, I think. I might be misquoting him. I’m playing a game of telephone. But basically he called out the obsession with TDD as the only way to write software. And he said that that’s not true. It can help, but it can also be taken overboard. And I just wondered what you thought about that as someone who’s an expert in testing. JULIE:  I think you could probably take anything overboard. Yeah, I don’t think that you have to have a test for each piece of functionality before you ever start writing the functionality. No, I don’t prescribe to the fact that your hands should be burned if you ever write a test that doesn’t pass the first time. But I think that overall, the idea of test-driven development, at least in my personal experience, really does produce better thought out APIs, code that runs better. And I’ve definitely seen and personally made the mistake of a lot of times writing code first, writing the tests after, and the tests aren’t good. The tests don’t actually ever fail, for example. I’m sure many of us have had the situation where we’ve written a test that cannot fail because it doesn’t actually test what you think is testing. JAMISON:  I just thought that meant your code was amazing. [Laughter] JULIE:  It’s so good, it’s proof. JAMISON:  Yes. CHUCK:  Yeah. JAMISON:  Infallible. JULIE:  But sometimes you just got to get something done. And I think sometimes you want to be able to, there’s a difference between prototyping and writing your final version of code. And you need to be able to sometimes be in a space where you’re prototyping and you’re getting immediate feedback about what’s happening. And sometimes, you want to loop that really, really quickly and you don’t want to stop and write tests in between. And I think that’s okay, as long as you know that you’re going to set aside the time later to do it. JOE:  Yeah. First off, I think I’m really surprised that DHH would say something polarizing. [Laughter] JOE:  But I also would like to kick in my two cents and that is that I think that TDD has a ton of problems and a lot of weaknesses. And it’s really, the worst software practice out there, with the exception of every other. [Laughter] JULIE:  Writing software is hard, man. JOE:  It is. It truly is. And so, I think that maybe at some point we’ll discover something better. And I have no idea what DHH actually said. But for me… JAMISON:  I don’t either. I could be wildly misquoting him. JOE:  Right. JAMISON:  But that’s the vague impression of the grumblings on Twitter gave me. JOE:  Right.  I don’t know. I just have this thing. If I see a shop that isn’t testing, then that’s an indicator that something’s wrong. If they’re not TDD-ing, there’s something wrong. Now, that doesn’t mean that they might not have good reasons. But to me, that just is my litmus test is to, there’s something wrong. It’s like a code smell, right? JULIE:  The other piece that I would add to this is, and I’ve said this before at talks, that testing isn’t something that you do because it is good to be testing. It’s not your lettuce. You do it because it increases your confidence in your code. And it increases your ability to release quickly and make changes without worrying that something is going to break. And so, if you’re paranoid like me, that’s a good incentive to do testing. And if you have a whole lot of self-confidence, maybe you’re just a really great coder. But maybe it’s a little bit more likely that you’re going to make mistakes and you might pay for it a little bit later. JAMISON:  The pot did not get very stirred. [Laughter] JAMISON:  I’m so bad at controversy. If AJ had brought that topic up, I feel like we would still be talking about it. JULIE:  Someone should take the stance that you should never write a test ever. CHUCK:  Yeah. JAMISON:  Yeah. Tests are for suckers. AJ:  I totally agree with Jamison on this one. JAMISON:  Yeah. [Laughter] AJ:  I don’t know why you waste your time writing code that isn’t getting anything done. JAMISON:  Unless you get, well I get paid by line of code. CHUCK:  Oh, wow. [Laughter] JAMISON:  I’m incentivized. AARON:  In that case… CHUCK:  I wish I had that problem. [Chuckles] JAMISON:  I just copy-paste jQuery a lot. CHUCK:  Comments are code, right? [Laughter] JAMISON:  I write sonnets to my code reviewers. JULIE:  Actually, on the topic of comments though, one thing that I heard recently from Igor and the Angular team which I really liked is when he goes through a pull request for Angular, he looks at the description, figure out what it does, and then goes to the tests before going to the code. And that’s using tests as comments, using your tests as comments and examples of how an API should be used in documentation for what your change should do. And I think that’s a great way of thinking about it. And that makes the time that you spend writing tests more valuable. You’re packing more into it. It’s not just, “Oh, I got to go write my tests and it’s going to take hours.” It’s, “I’m writing my documentation, showing people what stuff is doing, example, in my tests all at once.” That’s very efficient. JAMISON:  That sounds wise. AARON:  I’m interested in what’s your next big project, Julie? What’s the next thing you’re going to work on? JULIE:  I’m not really sure. I think that there’s still a lot to do with Protractor. There’s a lot that we can do to make it simpler, make it more stable. And really importantly, I’m not sure if this counts as its own whole project, but I think something that is a large task is making debugging easier, because debugging end-to-end tests is super hard. And there are a lot of things that could go wrong. That’s why it’s an end-to-end test. But it’s really hard right now to tell, did the framework mess up? Did the mess up come on my front end? Did the mess up come on my back end? Is this just a flake? Or is this something real and persistent? So, trying to figure out ways to make debugging easier is really the next big topic at the front of my mind. And there are some things that are starting to come out of this. So, recently I introduced a beta API that’s browser.pause() into Protractor. And what that does is when your test gets to that point, it will start off the Node debugger and give you a command line interface for dealing with it. And that lets you step forward not by line of code but by the next command that WebDriver is going to run. So, it gives you a little bit more insight into what WebDriver is actually doing as opposed to putting you into a debug point that’s in a section of spaghetti code that’s inside WebDriver, inside Protractor, inside your test, that doesn’t really mean a lot to you. And so, making that better, letting you maybe have an evaluation loop from the command line that you get there, these are all ideas that we’re working out to try to make the debugging experience better. AARON:  That’s awesome. JULIE:  And then as you mentioned, changes that are going to have to happen for Angular 2.0 and seeing what we can do even better with the fresh start that 2.0 provides. AARON:  That’s awesome. I’m excited to see it. CHUCK:  Alright. Well, if there aren’t any other questions or topics of discussion, we can go ahead and do the picks. JAMISON:  I think you covered everything so thoroughly that there are no more questions. JULIE:  [Chuckles] Awesome. CHUCK:  Awesome. Alright, AJ what are your picks? AJ:  I want to pick Cake. JOE:  [Chuckles] JULIE:  The band? CHUCK:  CakePHP? AJ:  Both kinds. Both kinds. No, not all three. Just the two that immediately come to mind. What Julie said, which is the band, first and foremost. AARON:  And then dessert. AJ:  And then birthday cake. JAMISON:  You were eating a hotdog when this started, too. There is a food theme today [inaudible]. AJ:  It wasn’t just a hotdog. It was a hotdog from a microwave. [Laughter] AARON:  It was gourmet food, in other words. JOE:  That’s one step away from a hot pocket. AJ:  No, no, no, because I went to my girlfriend’s family’s house for Easter and they had… CHUCK:  And they have hotdogs? That is sad. AJ:  They had lots of hotdogs and hamburgers. And some crazy lady had the audacity to tell the brother that was cooking the hamburgers that they were too pink in the middle. And then you know what he did? He started cooking them all the way through. And that’s when I got a hamburger. [Laughter] AJ:  All the good ones were gone. So, I’m also going to pick hamburgers with a hot pink center. Maybe a hot red center, even. JOE:  I stopped believing the story when you said, “my girlfriend.” [Laughter] AJ:  That is just so kind of you. [Laughter] AARON:  Non-fiction. JOE:  I’m sorry, AJ. Seriously, dude. You were just recently talking about online dating and now you have a girlfriend. AJ:  Yeah, I didn’t meet her through online dating. [Laughter] JOE:  Suck it, eHarmony. AJ:  That’s right. I actually met her on a blind date two months ago. CHUCK:  Oh, I so held back my zinger right there. Alright, Jamison what are your picks? JAMISON:  I only have one. It’s kind of like a dad book. So, it’s called ‘Crucial Conversations’. It’s barely old and well-known, so I’m sure some of you have read it before. But it’s basically about how to talk about difficult subjects, basically how to listen and understand what people are actually saying and come to a good decision. And not let the emotions of difficult things that you’re talking about prevent you from having a good decision. That was a poor explanation. So clearly, I haven’t read the book well enough. But it’s really great. You guys should check it out if you haven’t read it. AARON:  Did you get the cards? JAMISON:  I didn’t get the cards. I don’t know. Maybe I got the budget version of it. AARON:  They make for a real interesting ‘Crucial Conversations’. JAMISON:  Maybe we should use those next time on our guest. [Laughter] JOE:  Yeah. JAMISON:  We should do it when we have a framework debate or something like that. AARON:  A framework war? JAMISON:  Yeah, but moderated by ‘Crucial Conversations’. AARON:  Right. You have to follow the principles or you’re out? JAMISON:  Yeah. Yup. Okay, that’s my only pick. CHUCK:  Alright. Aaron, what are your picks? AARON:  So, I’m going to do two picks. The first pick is a book I just finished reading and I thought was awesome. It’s called ‘Steelheart’. Hopefully you guys haven’t picked it before or I’ll feel stupid. JOE:  I think I have, but please pick it again. AARON:  Okay. JAMISON:  Just the title sounds a lot like a book about horses for adolescents. [Laughter] AARON:  Yeah, it is. It’s about teenagers and horses. JAMISON:  Okay. AARON:  No, it’s a ‘Heroes’, if you liked the TV show ‘Heroes’ it was a lot like that. JOE:  Except ten times, a hundred times better. Except it didn’t suck, how about that? AARON:  Well, I liked Heroes. So, I don’t know. Basically, if you’ve even sat on your couch and stared at the TV remote and you ever actually reached your hand out hoping it would come to you but it didn’t, this book is for you. AJ:  But it didn’t. Only if it didn’t. If it did, the book’s not for you. [Chuckles] AARON:  Yeah, it will seem too real if you did. [Laughter] AARON:  But for the rest of us, it’s a good book. The second pick is this. On May 3rd, there’s going to be a cool conference local to Salt Lake. It’s called DevFest Family and it’s a developer day for the whole family. And there will be classes for kids and for teens and for adults and for all experience levels. Basically, we want to get everyone a little more tech’d out. And the slogan is “The family who nerds together, stays together.” So anyway, I’m excited about DevFest Family. It should be pretty cool. So, those are my two picks. CHUCK:  Awesome. Joe, what are your picks? JOE:  Alright. So, I’ve got three picks. My first one is a board game that I played at SaltCON. It was actually a prerelease version. It’s called ‘Robots on the Line’. So, I actually played this with the creator of it. And it was just an amazing game. It has all these little pieces and you would just, your whole job is to assemble robots. And it was so much fun to play without actually playing the board game. Just playing or screwing off the pieces was totally fun. One of the best board games I’ve ever played. And they did their Kickstarter a little bit ago but it’s already ended. But he opened up an Indiegogo campaign as well. So, if you want a copy you can pick it up on Indiegogo. I’ll put a link in the show notes. My second pick is going to be another game. So, it’s a card game. This one I was actually super impressed with. I saw this at Comic Con in Salk Lake City. Me and one of the other hundred thousand attendees were able to walk by all these crazy booths. One of them had this game called ‘Japanese: The Game’. It’s a card game and it plays like a collectible card game like Magic or something like that. But the cards are actually the Japanese, is it called kanji, the writing. And in order to play the game you actually have to learn Japanese. And so, you’re playing a fighting collectible card game à la Magic. But while you’re doing it, you’re learning Japanese, which I though was completely awesome. AARON:  Anyone who knows me and has heard me with my Japanese accent would know I would probably love this game. JOE:  You would love this game. I was completely astounded by the idea. They’re going to do other languages. They say they want to do Korean and Chinese. But I thought that was just totally cool. [Chuckles] JOE:  That’s at JapaneseTheGame.com. And then lastly, for my third and final pick, I want to pick Julie Ralph. AARON:  Julie. CHUCK:  Aww. JULIE:  Aww. [Chuckles] JOE:  Julie is completely awesome. She’s super fun to go fishing with. And I think that the person on the Angular team who is the test person, that’s like, Angular is the testing framework. And then you’ve got the test person. I think that makes Julie the absolute ruler of the universe, just based on that. JULIE:  I’ll take the title. But I have to give a lot of credit to the rest of the Angular team for being awesome test people by themselves. JOE:  [Chuckles] And that’s my final pick. CHUCK:  I was going to say, if she’s master of the universe, does that make her He-Man? JOE:  She-Ra. CHUCK:  She-Ra? AARON:  She-Ra. JULIE:  [Laughs] CHUCK:  Alright. I’ve got a couple of picks here. The first pick is a conference. I’m actually going to pick a couple of conferences I’m going to be speaking at. The first one is OpenWest and it’s here in Utah. It’s in a couple of weeks. And I’m going to be talking about SOA, which is service oriented architecture. So, I’m going to be talking about that. Then I’m going to be doing a Ruby on Rails workshop. So, if you’re interested in either of those, or if you just want to come hang out, that’d be awesome. I’m also going to be speaking at Agile Roots which is another conference in Salt Lake City. And that one’s in the middle of June. And you can find that one at AgileRoots.com. So, check both of those out. And then I’m also going to pick one. I usually don’t pick the sponsors, but this one is really wicked cool. It’s called WatchMeCode. And it’s a video series by Derick Bailey who’s been on the show before. He’s the author of Marionette which is the framework that sits on top of Backbone. And just some awesome JavaScript content there. So, I’m going to pick that one as well. Julie, what are your picks? JULIE:  Alright, I got two. So, the first one is HabitRPG, which is HabitRPG.com. And the basic idea is you set up a list of habits that you, good habits or bad habits, and tasks that you want to do every day. And you’ve got a great little pixelated character. And you get experience for doing your dailies and lose health if you have bad habits. And this is really great, because if you’re like me, working towards that little pixelated mace is so motivating. And so, I would be like, “Yeah, I’m going to go floss my teeth. [Chuckles] Let’s do this.” So, HabitRPG. It will improve your dental hygiene. And then my second pick is Test the Web Forward, which is at TestTheWebForward.com. It’s a W3C one-stop shop for open platform testing. And I learned about it really recently. And I feel like this is one of those things that I should have known about years ago. They’re trying to bring together tests for all of the web platform specifications and have a suite that you could run against a browser and be like, “Yeah, this is an up-to-date browser running all the specs correctly.” So, I’m reading through that. I think it’s got some really great stuff. And I think specifications need to be more aware of how what they create is going to be tested. And so, I think this is a super important effort. And I’m really excited to read more about it. CHUCK:  Wow. I’m looking at HabitRPG. Dang, this is going to totally mess up my non-productivity. JULIE:  It’ll make you so much better. CHUCK:  Awesome. Alright. Well, that’s all we’ve got. If people want to know more about you or Protractor, where do they go? JULIE:  Look at Protractor on GitHub or I tweet @SomeJulie. CHUCK:  Alright, cool. Well, thanks for coming again. And we’ll catch you all next week. [Hosting and bandwidth provided by the Blue Box Group. Check them out at Bluebox.net.]  [Bandwidth for this segment is provided by CacheFly, the world’s fastest CDN. Deliver your content fast with CacheFly. Visit CacheFly.com to learn more.] [Do you wish you could be part of the discussion on JavaScript Jabber? Do you have a burning question for one of our guests? Now you can join the action at our membership forum. You can sign up at JavaScriptJabber.com/jabber and there you can join discussions with the regular panelists and our guests.]

Sign up for the Newsletter

Join our newsletter and get updates in your inbox. We won’t spam you and we respect your privacy.