055 iPhreaks Show - Faking Out Backend Services

00:00
Download MP3

The panelists discuss faking out backend services.

Transcript

[Would you like to join the conversation with the iPhreaks and their guests? Want to support the show? We have a form that allows you to join the conversation and support the show at the same time. You can sign up at iphreaksshow.com/form] CHUCK: Hey everybody and welcome to episode 55 of the iPhreaks Show. This week on our panel we have Ben Scheirman. BEN: I haven't slept for 48 hours, because that would be too long. CHUCK: Jaim Zuber. JAIM: I love Steven Wright# BEN: That’s Mitch Hedberg. JAIM: Oh, it is? BEN: Yeah. JAIM: Oh. CHUCK: Pete Hodgson. PETE: [Laughs] Good morning, from Berkeley, California. CHUCK: I'm Charles Max Wood from DevChat.tv and this week we’re going to be talking about faking out the back end on your iOS stuff. Pete’s the expert here, he recommended it. So Pete, how a lot are we talking about? PETE: Yup, because I'm an expert. CHUCK: Somebody called me a thought leader the other day; I was really confused. PETE: That’s offensive, because it’s like, “Is someone behind me?” [Laughter] is he talking to someone? So let’s start with a why would you wanna do this, maybe? If you're testing your application, your app normally talks to a bunch of backend services. Most apps, I think, talk to at least one backend service – that might be your own that you're building, or it might be one that’s a third-party service that you're using like Paz or StackMob or something like that, or Twitter or Facebook [crosstalk]. CHUCK: I thought you were talking about Facebook and Facebook. [Crosstalk] Don’t they all talk to Facebook? BEN: Actually, StackMob got acquired by PayPal; I think they're shutting down their stuff. PETE: Weird. Okay, well if you're integrating with PayPal or Facebook - [inaudible] two options I suppose, now. MySpace could be integrated; I mean MySpace – they're big in the music scene. Anyway, you need to talk to these backend services, right? And when it comes to testing things, I either manually test things or do test automation. Sometimes you wanna fake out those services, so you wanna replace a real Facebook with a not-real Facebook. BEN: I wanna do that all the time. CHUCK: [Chuckling] Yeah. Fakebook. Do you wonder if that domain’s available? PETE: I guess not. There's different reasons you wanna do this. Let’s say, you want us to simulate – sometimes it’s just for convenience, like let’s say you wanna simulate a user that’s been locked out of their account, so you wanna test like you're working on some new UI for what you're going to show to the user when they try and login and they're locked out because they haven't paid up for this month or whatever. You could kind of go over to the backend system and set up the user that’s been locked out and then test something, and then remove that user or whatever. Sometimes it’s easier just to have a fake version of the backend, then you can just kind of twiddle some knobs and say, “Next time this has to login, just pretend it’s a fake user rather than a real user.” Or sorry, just pretend it’s a locked out user rather than a regular user. So that’s one scenario. CHUCK: So this is mostly in testing then, right? PETE: Yeah, I think so. I mean, I get, though, the other scenarios, but I can’t [inaudible] many. BEN: I mean, there's a lot of scenarios where the scenario that you're trying to reproduce is somewhat difficult to do manually, and if you do do it manually, there are probably some steps you would have to take to set up the world in such a way that you can accurately test that, and then you test it, and then all that work is sort of gone because you can’t test it again without recreating those manual steps, right? PETE: Yeah. It’s inconvenient to do that when you're doing manual testing, so it’s still beneficial when you're doing manual testing, but once you're doing automated testing it gets really, really [inaudible] to try and automate all of those manual setups and tear down steps. Sometimes you can expose an API that will do it for you, but it’s always super duper painful. I think that’s when most people start seeing faking out services is when you're doing automated rather than manual QA. Yeah, it’s painful, right? It becomes very, very tedious. If it is your full-time job to be a QA in particular, it gets to be very tedious to be doing this stuff over and over again. JAIM: [Inaudible] one of my first jobs after college, I was working on this ASP.net application. I was working on a screen that was five or six screens deep into the app, and I was just clicking through, clicking through to get to the thing that I needed to test, and then I clicked a button to run a report and then I would have to make my change and do all that all over again. It was particularly bad because the main body of the app was in an iframe, so it’s not like you could direct link to it very easily. The feedback cycle on that was just ridiculous. PETE: This is assuming you're testing your own internal services and you can actually do that. If you wanna test – unless you're doing a Twitter app and you wanna test what happens when someone blocks another tweet, maybe you can kind of simulate that scenario, but there's actually some scenarios that you literally cannot simulate unless you have access to the system. Date is kind of the canonical one, like what happens on a leap year? Does it render February 28 correctly? Wait, is it 28 or 29? CHUCK: 29. PETE: See? That’s why you need testing. CHUCK: [Chuckles] Because Pete can’t do dates. PETE: One date in a program. [Chuckling] A few years ago I was working on an iOS app for a bank and we needed to check what happens when a user has $10 million. Does it fit in the screen correctly? BEN: So as a tester, you just need to get a $10 million dollar deposit. PETE: That’s correct. And really, for a while, that’s what some of the QAs did was they would go to the backend systems and do the equivalent of transferring $10 million in a wire transfer or something like that, like do all of these crazy things. Especially like these huge enterprises like banks, the system that you need to actually make that change is four continents away – literally continents away in terms of the team working on it and all the rest of it, so it’s very hard to discover even what system to use to simulate these situations. All of these things point to using fake versions of the services where it’s appropriate. BEN: So I think we’ve sold everybody on the, “Yes, that’s probably ideal. How do we do it?” CHUCK: CBT with mocks. PETE: No mocks. Actually, yeah, you can do it with moss. When I talk about it, I kinda lump this into two main categories – what I call end process and out of process – process as in operating system process. You can fake things out in process, so you can actually use things like mocks or use a kind of a custom-built thing and replace your real network layer with a fake network layer. JAIM: Back up! What is a mock? PETE: Oh. A mock is a shorthand term for – well, what's also named as a test-double. It’s a kind of a class that looks and smells like a real duck but is actually a rubber duck, so it’s a fake version, or it’s a not-real version of something that you're replacing the real version with for testing purposes. CHUCK: Yeah, so usually –. JAIM: [Crosstalk] Going to the Internet, I've got some Internet type thing that I can control to my own whim. PETE: Yes, exactly. And usually the way you use these things is you kind of say, in your test code, you say, ‘hey, next time you make a get to this URL, then please return this JSON payload’ or something like that. Or, ‘next time there's a post to this URL with these [inaudible] parameters, make sure that you return this error condition or something’ – something along those lines. That’s normally how I've seen those things used and then sometimes people build a nice API on top of that that’s kind of domain-specific, so rather than saying, ‘when you go to this URL, return this JSON payload’ in your test you can say, ‘when the user logs in, simulate that they're locked out of their account.’ JAIM: Very nice. So just to pick nits, that’s a stub, though, correctly. Right? So people aren’t calling in furious? PETE: Yeah. CHUCK: So a mock is a full-on fake out object. For example, if you had something that acts like the network layer in your application and it exposes the right interface so that you can call it like you call the network layer, and it gives you answers like you would get from the network layer but is not the network layer – that’s a mock. If you take the network interface class or you override or otherwise change the behavior on a particular method or function, so that it still gives back the right answer but it doesn’t actually do all the work, then that’s a stub. PETE: That’s in the terminology of – I'm guessing you're getting that from RSpec or one of the testing tools in the Ruby world. I sighed a little bit when Jaim said that because I don’t actually think there's a very good definition. There are several competing definitions. Chuck, I've heard the one that you said before where mock is like you replace the whole thing and stub is you replace part of the things. In different programming communities they use different words for that, so that thing that you called a stub may be would be called a partial mock or a live mock because you're modifying the real class. Anyway, what I'm getting at is there's actually not – I don’t there's a very, very clear definition. The way that I think about it is you use a mock when you wanna check how the thing you're testing interacts with the thing that you're faking out. So you wanna check that it calls things a certain way, so you're testing the way that your system uses the thing rather than faking stuff out. A stub is when you just wanna fake out that thing so that you can inject data into your test, which I think is what Jaim is getting at. But it’s complicated, or it’s unclear in my mind. [Crosstalk] BEN: And if you were following the Is TDD Dead? fiasco recently – if you haven't, watch DHH’s keynote for Rails Conf and then watch the hangout he did with Martin Fowler and Kent Beck. I bring that up because both Martin Fowler and Kent Beck both said that they don’t use mocks if they can avoid it, which is interesting. When both of those guys say the same thing, you should take note. PETE: Yeah. I think a lot of those early [inaudible] didn’t really – I mean, mocks are fairly mocking frameworks and kind of extensive use of test doubles is a relatively more recent invention than unit testing itself, the automated unit test as a concept. And what’s-his-face talks about that – I don’t remember who it is. I think it was Gary Bernhard, was talking about that in his post, his response to DHH. He was saying that if you look at the timelines, actually, they're not quite the way that DHH says [inaudible]. But anyway, slightly off-topic, I guess. CHUCK: Yup. Going back to faking out the backend, you said that you don’t necessarily have to use mocks, or are you –? PETE: Right. CHUCK: [Inaudible] curious anyway. PETE: There are two levels of that. You don’t have to use a mocking framework; you can kind of use a thing that actually full-on replaces it, rather than you mocking out methods, something that full-on replaces the network stack. But then the other kind of category of tools you can use if you don’t wanna use in-processed things like test doubles is to basically do out with process, fake services where you actually have a separate thing running somewhere; a separate process that’s listening over HTTP – assuming you're faking out HTTP – and it’s listening on a port just like a regular web server would do, and you configure your app to talk to that server rather than the real Twitter or the real internal service. And then you can talk to that – you normally would have a back channel or side channel kind of API that you use to talk to that service and say, ‘hey, someone’s about to send a get to you on this path. When that happens, please return this response.’ A very similar idea, but done out of process. And there's a lot of tools out there, out of the box, little services that you can use to build these things. JAIM: So what does the out-of-bandwidth conversation look like? PETE: It depends a lot on which tool you are using. Some of the tools – there's a few different techniques that I've seen used for these fake services. One of them is a record and playback approach, where the fake service will act as a proxy and it will route all of the traffic through to the actual real back-end services. It’s essentially doing a man-in-the-middle attack on Facebook or Twitter or whatever, or your services. By default, it would just be recording all of these interactions and just saving them to disk or something. At a later date when you want to simulate a different scenario, you can go back through those recordings and say, ‘hey, rather than just passing this through to the backend, this time I want you to play back this recording, so I want you to simulate this thing that happened before.’ And that was quite popular with manual QA whereas [inaudible] we used a lot with it when I was working at this back. They had this rather large fake service toolkit and one of them – it had this whole UI and you could login and setup different scenarios and give them names and modify the payloads they were returning and do all sorts of fancy stuff and that was mainly used by QAs who were doing manual testing, so they didn’t actually have to actually get $10 million into their bank account or whatever. JAIM: Okay, that sounds like some – that’s pretty cool because they're actually hitting a network and actually getting a valid response that you might get from a service. PETE: Yeah. JAIM: It sounds kinda tedious to set up and keep running and make changes to. PETE: Yes, it is. It’s a little bit more clunky; it’s an extra moving part, and it’s quite fiddly. If you do a good job then it can be as easy to use in your tests as in process stuff. Although the mechanism is more clunky, you can kind of abstract that over with a nice API if you're willing to put the time into doing that, so yeah, there's some tradeoffs there. But the benefit is, it’s a lot easier for manual QA to use – that’s one thing. You can build an extra API on this thing whereas it’s quite hard to do that if you're doing all the stuff in process. It lends itself more to manual testing – that’s a big win, or that’s useful if you're doing that. And the other thing –. BEN: It seems to me like you're talking about some benefits; there's definitely tradeoffs where you get a more realistic, like there's actually data going over the wire, you have to wait for the response – things like that – so you're actually exercising network code and maybe reachability code as well. Whereas if you do the stubs, like in a unit test, or say, an automated test, it might be advantageous or desirable to not hit the network so that you're testing whether or not your office has internet connection, but instead just testing that you made the request, you got the response, you parse it – the JSON – correctly, or whatever, and were able to act on it. And you could do that much faster if you can stub it at the network layer, right? PETE: Yeah, [crosstalk]. BEN: Which I think is desirable for a large test suite to be able to run quickly and without the intermittent failures that might result in having a congested network, or maybe somebody reset their router during the test run or something like that. PETE: I don’t see that to be a problem that much, actually. I agree there's other reasons to do it, but I haven't seen reliability be that big a deal because you're running the server on the same machine. If you're doing it in the simulator then you're running it on the same machine. I guess if you're doing a lot of testing with physical devices then maybe it [inaudible]. JAIM: Okay, so you're actually running a service on the actual local machine? PETE: That’s the way I've normally done it. Sometimes, if you’ve got very non-technical QAs, then they’ll be like a shared service because they don’t know how to use the command line and start up a service and keep it up to date and all that kind of stuff. CHUCK: Yeah. I wanna go back a little bit to the recorded responses. I've wound up doing this quite a bit lately with the client that I'm doing work for, and it is really nice because you get real data back that you can work with, and you get a consistent response. So if they're down or if the network’s down like you guys have said – one thing that I did run it though is that if I made the request and there was even something just a little bit different with any of the parameters or anything, then I would get the error saying, ‘I don’t have a recorded response for that.’ PETE: Yeah, so normally what you can do is you can configure the tool to have wildcard parameters. And I think if you're using VCR or something like that in Ruby world you can do the equivalent thing there or Webmock. CHUCK: Yeah. BEN: Right, I was actually going to pick one of these; I’ll just paste in now the VCRURLConnection by Dustin Barker. It’s basically what you'd expect if you're familiar with VCR in Ruby land. Basically, it looks for a cassette – a recorded cassette file – and if it doesn’t find one it makes a request and if it does find one it just returns the canned response headers and –. They're not canned – they're actually what you got last time. So that kinda gives you the best of both worlds, like if you're calling – I don’t know – Twitter to get a list of followers. Once you have that response, you don’t need to make that call every single time, and so to me, it seems really nice to have both a real response – so you know that that request made it through once, at least, successfully – and then periodically you can delete your cassettes and rerun them to make sure that the service hasn’t changed or whatever. I haven't used VCRURLConnection yet, but I heavily use VCR to keep my tests from hitting external APIs in Ruby. CHUCK: Yeah. The other nice thing with the changing of the parameters is that if you do wind up making changes to your [inaudible] or around your service, and it does wind up formatting something different, or winds up getting a parameter if you're doing more integration-style testing where something else calls into it and so it’s getting something that it shouldn’t, then you get those errors and you get your red flag. ‘Hey, I'm changing something and I'm asking for something different from the service.’ So it can work both ways, but I found ti a little bit brittle in the sense that if I'm writing a new test, I have to get those exactly right. They're super handy. JAIM: So we talked about the out-of-process test. What are some of the in-process tests? How does that work? BEN: Without using any other tools, except unit test tools. You could use something like Kiwi or OCmock. You can create a layer for all of your network interaction yourself and then mock out those requests and responses. That’s kind of tedious, but you have ultimate control over how you want it to work, so you just say, ‘Okay, you should receive this selector and when you do, you're going to return this result.’ It gets a little bit tricky when the result is actually a callback block that you pass in, so something like, ‘fetch widgets with completion,’ and the completion block accepts as an argument the array of widgets that you received, or maybe the NSData from the response – however you got that set up. It’s not as easy as mocking that selector and returning a result because there's no return value anymore, right? You have to grab the block. And there's a couple of ways to do this: one of them is to use spies – and spies are kind of a complex topic but basically it allows you to intercept arguments past your method. A good way to do that with OCmock is to – there's basically a way to verify arguments. You might verify that this method was called with the string ‘hello’, and that’s like an exact check. But you could say, ‘this argument was called with anything that’s not nil’ so I think that’s OCMArg any, which is kind of a special object you place in and will validate that argument. So using the same technique, there's one that says ‘verify with block’, so it’s OCMArg verify with block. You specify a block and your block takes in the argument that was passed to the method. Now you can do stuff like maybe if you're expecting a string length of greater than five, then you would just do the check there and return yes or no whether or not it’s a valid argument. Using that technique, you just – your argument to the block is the callback block for that network call, so you cast it back to the block type that you're expecting, and then invoke it with the canned data. Does that make sense? PETE: It does [crosstalk]. BEN: It’s really hard to say without – like I'm shaking my hands all over the place. It’s really hard to say in words and it’s [crosstalk]. PETE: You're talking about asynchronous programming on a podcast is pretty tricky. JAIM: Especially with OCmock where the –. BEN: But if you get the notion that you can intercept the arguments passed to a method and verify them with a block, so I'm not really verifying the – you may be verified that the callback was non-nil, but what I'm really trying to do is get a handle on the block passed into it so that I can invoke it with my own canned data in the test. And so I’ll just do that: I’ll just get a reference to the block, I’ll invoke it, pass in my stubs data – whatever I want my response to be, say, it’s like an empty list of widgets, I just return NSArray array – and then return yes, saying that the argument was valid. So I'm sort of piggybacking on that argument validating system that OCmock gives you. And there's this [crosstalk] to do this with Kiwi because this concept of being able to – it’s easy when the arguments are well-known, but what if it’s a guid from a new customer that you can’t possibly discern in your tests that it gets created somewhere else? So you may just wanna say, ‘oh, it’s a string in this pattern’ or ‘maybe it’s a string that’s got a length of greater than one of something,’ so you can always make exact comparisons on the arguments. PETE: I've actually got a code example. I did a workshop, an iOS automated testing workshop a few years ago, a year or so ago, and we had an example that was pretty similar to what Ben is describing. But what I normally try and do in those cases is have, rather than mocking out at that very, very low level, I will make an API client class, like a custom-written client class that represents that service. In the example that we had, we were simulating this BART– I was writing a BART app. BART is the public trans system in the Bay Area and it’s a little app that will tell you when the next BART is coming and you need to fake out when the next BART is coming so you can test it. So we had a BART client, I think, that would expose that API and it had that same asynchronous block stuff Ben was talking about and we made a spy but it was actually a lot easier just to actually hand-roll a spy. So we just made a fake version of that BART client class and whenever you called it, rather than it actually doing anything, it just kind of grabs the block that you passed in and then the test could then go back to that spy and say, ‘hey, pretend that this thing happened.’ So, ‘hey, you were just called with – the real code just called you and said, ‘make this request and then when you're done, call me with the results.’ Now, I want you to pretend you're done and return to that client with these results.’ So it’s a very similar idea but it’s pretty much the same idea, just –. BEN: Yeah, this definitely looks more explicit and when you either don’t know how to –. For instance, if you didn’t know you were looking for the word ‘spy’ because that’s like the well-known term, it’s good to know that you can roll this stuff yourself and in languages where you don’t have nearly the dynamic stuff that we do in objective-C and Ruby, sometimes this is required. It’s kinda funny when I see that people are doing TDD with C or – [chuckles]. It’s like, “How do you even do it?” People do it, but you have to get creative and this seems like a really explicit implementation, which is pretty easy to understand. It’s just more code than the spy would be. The thing is when somebody comes across the spy code, the OCMArg would verify with block and then the thing that I was talking about – it’s not obvious, what you're doing and why, so that’s the only downside, I think, to doing it that way, is it’s easy to set up but then when you come back and read it later you're like, “What is this doing again?” And you scratch your head a little bit. PETE: Yeah, and I think that was part of my motivation of doing it this way was because it was, I was at a workshop so I was teaching people how to do this stuff, so I didn’t wanna – I wanted to get the concept across of the spy and not have to actually explain the crazy, do the equivalent of that handwriting stuff that you were just doing anyway. Then there's this thing that you verify with this thing [inaudible] - sometimes it’s easy if you lay it out in [inaudible]. BEN: Actually, I have an example of that online. I will link to it – exactly that – so that way, people listening in their car can go back and look at it later and know what I was talking about. JAIM: Pete, I found that to be a pretty powerful pattern. So what you're doing is you're taking – you’ve got your BART client, which normally goes to the BART service and gets the schedule. Is that right? And you're sub-classing that, so you're creating a drive class and when you call the [inaudible] schedule with block, you're just returning a canon response of whatever the schedule is. Is that right? PETE: Mm-hm. Almost, but not quite. Because it’s asynchronous, when someone calls get schedule with block, I don’t do anything. The fake version, the spy, doesn’t do anything at all. All it does is it grabs that block and keeps it for later, and then later on the next step of the test will say like, ‘okay, now I want you to simulate this schedule coming back.’ So it’s almost this – what you're saying, Jaim, but it’s the extra complication of it being asynchronous, so it won’t return straight away. Funnily enough, I talked about this in quite a lot of detail on Chuck’s JavaScript podcast because I was on the other one of Chuck’s podcast. I was on that podcast talking about how to do this asynchronous testing in JavaScript where it’s a lot more of an issue because everything’s asynchronous, so it’s an interesting parallel. It would actually be really nice if objective-C had a nice promises library; it would make all of this stuff way, way easier, but unfortunately that’s not the case. BEN: I’d be curious what one looks like if the language even supports it. It’s one of the things that I'm sort of envious of C# about is the Await keyword, which allows you to write code, and I've never used that. I've just seen examples of it, so maybe I'm getting some of the details wrong, but it allows you to write code that looks somewhat imperative. Just step by step, you know, call this, call that, and under the hood it gets rewritten to be async with callbacks, but you don’t have to nest blocks everywhere to get that [crosstalk]. PETE: Yeah, it would be awesome if –. You know what? I would not be that shocked if Apple did something like that. Given their history of doing crazily impressive stuff with clang and doing, like [crosstalk]. BEN: The thing is, there's no hiding it. Clang is open source, so you would see the tea leaves at least a year in advance in the open source mailing list. I saw the – I have a friend who reads the forms and the release notes and all the commits. He’s an interesting – he has a unique interest, I'd say. But anyway, he sent me a link to the module stuff and I was really thinking – I even read the whole spec and what it was trying to do and my initial thought was that it was going to be name spaces for objective-C and I was like, “Yes, that's going to be great!” Turns out name spaces are really, really difficult to implement in objective-C and may never come, but it turns out the module was more of a header optimization strategy. In practice, in Xcode, you can just say @import module name, like core location, and it imports the headers and it will automatically take care of linking the framework for you as well. PETE: That’s awesome. BEN: But it’s more of a header optimization thing. But yeah, it is really nice. If you think you need core data and you just say @import core data, no quotes, and –. PETE: Oh, wow. BEN: Yeah, it’s pretty cool. But you have to be able to understand how – what these features mean when they get proposed and discussed on the mailing list, it’s not always obvious. I will say, though, if we saw some new keywords, I think that would make news pretty quick. PETE: Well, I know there's a lot of Clang developers that listen to the podcast, so hopefully one of them will take our idea and run with it [crosstalk]. BEN: I think 100% of the Clang developers. PETE: That’s what I've heard. [Laughter] 56.7% of statistics are made up on the spot. So one thing I did [inaudible] do more as a brain dump than anything else is to talk through some of the tools that are out there in the out-of-process world. One thing that I didn’t mention of the advantage of out-of-process mocking, so having a server actually running, you can share this across multiple platforms. So if you're developing [inaudible] an Android app, for example, then generally your APIs are obviously going to be saying that that thing is using, so it can be quite beneficial to do out-of-process. There's a few different tools out there that I've used that I like a lot. Luke Redpath, the inimitable machine of productivity that is Luke Redpath, has this Ruby gem called mimic, which is essentially a little Sinatra app plus some extra stuff to make it easy to build fake services, so that’s a good one. I’ll put all of these in the show notes once I'm done talking about them. It is like 17 of these that have been built by ThoughtWorks, because we’re kind of obsessed with building testing tools, and then rebuilding them, and then rebuilding them – maybe part of our downfall as well. One that isn’t by ThoughtWorks is this thing called mockey. This one has a very rich user interface and has lots of the record and playback features. It’s the one that they were using at this bank and it’s very, very rich in terms of usability. Then there's the ThoughtWorks one called Moco, which I really like. You can either run that in-process if you're a Java person, or out-of-process if you're not. There's another similar one called stubby4j, which is pretty similar. And an interesting one is this thing called – I don’t know how to pronounce this; it’s called mountebank, I think. It’s a similar idea of an out-of-process fake service, but it has this extra kind of thing that it will actually fake out not just HTTP but other protocols as well. If you're doing a lot of SMTP, for example, which you'd probably not going to do from an iPhone but if you're using a lower-level web sockets library or some low-level TCP thing, then you can use mountebank to stub out those protocols, to fake out those protocols as well. The mountebank website is quite good because it has some quite nice diagrams that explain the concept of what it’s doing. So those are some good tools that are out there to look into if you're interested in the out-of-process world. JAIM: Pretty cool. So what are some other approaches for the in-process stuff? We talked about creating spies and sub-classing our clients. I've heard about people actually going through and stubbing out NSURL responses, so even if they're using AFNetworking, they're actually going in and stubbing out responses so they test the entire stack of mapping to the objects. Have any of you guys done that? BEN: Yeah, I played around this one library called OHHTTPStubs and this will hook into NSURL connection, NSURL session and any other frameworks that use those two, such as AFNetworking. It wouldn’t work if you're using some homegrown BST sockets-based networking system like ASIHTTPRequest, so migrate off that library if you're still using that. But yeah, so pretty much any networking library, including hand-rolled ones will work with OHHTTPStubs and the idea is you just say, ‘okay, you set up a stub and then you make the request, and they you give it a response.’ It actually gives you an NSURL request and you are responsible for giving it that response. So it seems like this is probably the most common I've seen – I mean, it’s got a lot of stars on GitHub; I don’t know of any others. JAIM: Okay, cool [crosstalk]. PETE: The other one that I've heard of before is Nocilla, but I don’t know – I'm looking at it [crosstalk]. BEN: Oh yeah, that does sound familiar. PETE: Have we had [inaudible] on the show? BEN: No, but we need to. PETE: Yeah, he’s actually a really good guy to talk to about this stuff because he does a lot of stuff. He has a lot of good opinions around networking and APIs and stuff like that; we should get him on the show, actually. I think he pointed me to Nocilla. BEN: Nocilla will work with ASIHTTPRequest. PETE: Perfect. BEN: You should still migrate [crosstalk] PETE: [Crosstalk] you can use Nocilla. JAIM: I still run across code that use VSI. Like, really? The first thing we do is get off this. BEN: It was such a great library back in the day; it’s just unfortunate that the developer decided to leave. He was a great guy, by the way, he’s really, really helpful but got a lot of hate because he wrote this thing to support his own needs back when you didn’t have a good API for network requests. And that didn’t become, in my opinion, a usable network library didn’t come until iOS 7; the NSURL connection was just way too clunky. The fact is, it didn’t build on top of the NSURL loading system, so any improvements Apple made to performance or bug fixes or whatever, you can’t benefit from. In addition, there's libraries like these that hook into NSURL connection and NSURL session and so again, we benefit by sitting on top of those things to benefit from the work of others. JAIM: Free software isn’t – free isn’t toilet. I put a toilet in my front yard, but don’t complain to me if you clog it, you know? You can use the toilet – yeah, it’s a problem. But no, it was a great library. It’s like, ‘Oh, this is hard’ and went on doing something else. PETE: It’d be kinda nice if AFNetworking had something built into it, it just occurred to me. Because that’s like [crosstalk]. BEN: I mean, it’s kinda against the mantra of that framework is to not be gigantic and not be a bloated thing with so many features. In the latest AFNetworking 2.0 branch, they removed features and pulled them in a separate sub-specs in Cocoa pods. When you pull in AFNetworking, you can pull in everything that you had before, but they're actually separated so you can just pull in – I forgot what it is – the core. So if you don’t want things like the UIImageView category for loading an image from URL, you can not pull that in, which I think is good. I like that there's always handy tools, but you don’t have to pull them all at once; it’s kinda nice. There's a list of AFNetworking extensions – there's a lot of them, actually, so you can take a look in here and see. There's some official extensions like AFJSONRPCClient or AmazonS3Client to a bunch of third-party ones like Gravatar clients and CSP Request Operation – things like that. PETE: That becomes interesting actually, and I've had people who’ve asked me quite a lot when I've talked about these faking out services is ‘what do you do when it’s not you're code – you're not the one talking to the service; it’s some third-party library?’ So you're using the Gowalla API or the client library for MySpace – the iOS client library for MySpace. You’ve got two options at that point: you either do this kind of shimming thing where one of these tools that actually fakes out, kind of sticks itself in between the low-level network in any of your code, and then you're stuck with having to figure out what the API actually looks like, or you can fake out the client library which is normally a better choice, I think, in that situation. CHUCK: Yeah, I tend to agree as far as mocking out the client library goes. I mean, then you have a certain level of control and you can make the assumption that the client library then does the right thing and you can return something that looks like, or is the type of object that is going to be returned by the client library. PETE: And as Ben said earlier, the tradeoff there is now you’re doing less integration testing, so the scope of your test is smaller, which is good because it means your test is more focused, but it’s bad because if you don’t have something high-level, a broader scope test that’s testing the whole thing – including that client library – then you're exposing a risk. There's two big risks in these situations: one, the client library is crap and has bugs in it, which is not uncommon actually, particularly if it’s written by people who aren’t iOS experts, which is quite often the case when they have to build a client library for Android, JavaScript, iOS, everything else. The other big risk is that you're not using it in the right way, and you don’t know that you're not using it the right way because you're faking out your interactions with it, so there might be some subtle rule that’s very well-documented in the docs that says before you make a request to do this, you have to first make sure you’ve done this other thing. In your low-level tests, even if you're not doing that anything will work because you don’t know to check that, but then when you go and use it in real life, it won’t work in some situations. There is that tradeoff there that if you tighten your scope, then your tests would get more focused but it means that you're missing out on that integration level. That’s what bit DHH apparently, is he didn’t know that he had to write integration tests for his attachments in basecamp and that’s why he had [inaudible]. CHUCK: [Chuckles] JAIM: That’s it, right there! PETE: Go figure. JAIM: So we talked about mocking in iOS; we talked about OCMock which has been around forever. PETE: [Crosstalk] celebrated it’ 10-year anniversary. JAIM: 10 years – that’s pretty amazing. As Ben alluded to – some of the syntax can be pretty daunting. There's another library that I've used and kind of enjoy – I've used OCMockito, which takes its lead from the Mockito project in Java, which does a couple of things that I think are pretty nice, where it lets you put your assertions [inaudible] I totally forgot what it is, the benefit. I haven't done testing in a while. BEN: Like the Arrange-Act-Assert style? JAIM: Yes. So you do it – you run the test, then you put the what happens, so you can verify what you wanted to happen did happen. It also has nice mocks by default, so every time you change something it doesn’t break your tests. It’s a pretty cool library. BEN: So, nice mocks in other frameworks, the opposite of that is called strict mocks. If an interaction happens outside of what you specified, it will throw in an exception which can be useful like if you're writing a controller that would fire a missile – you probably only wanna do that once – or whatever, make an HTTP request once or whatever. Whereas with a nice mock behavior, it won’t fail if that method gets called twice or if some other method [inaudible] sort of a helper gets called in the process of doing whatever. I definitely prefer the nice mocks unless I want the strict mock behavior. [Crosstalk] OCMock – doesn’t OCMock behave the same way? JAIM: The default is strict, which you find out pretty quickly when you start breaking tests every time you touch the code [crosstalk]. But it’s pretty easy [inaudible] nice mocks, whatever. BEN: Again, it goes back to terminology. Do you know what a nice mock is versus a strict mock or a spy? Another one that we talked about earlier was partial mocks where you're mocking some method of an existing instance of a class. Rather than say, ‘here is a mock object’ that is its own instance and you're specifying behavior programmatically at test time, a partial mock would be like, ‘okay, I'm going to [inaudible] some other object that already has behavior and I want all that behavior except for this one method that does something side effect or dangerous-y. I'm going to mock out just that one method and return some canned value.’ JAIM: Partial mocks are very awesome. Unfortunately not in OCMockito yet; if you want that, you won’t put it in there. But they are in OCMock and I used them quite a bit, and they're great if you wanna step out something from the framework – you’ve got something that would be a singleton that you'd get from iOS. You can just stub it out. PETE: Yeah, I get what you mean. You got this big, old, chunky API and you just wanna fake out one part of it. JAIM: Yeah, some class method on something you can [inaudible] break out a partial mock and return whatever you want. PETE: I do understand why John doesn’t want to put them in there though, but I'm guessing he just doesn’t fundamentally agree with people doing that stuff. I've seen it really badly abused, where people just get really confused at what the point of mocking is, and they end up partially mocking the class that they're testing, and then they tested that partial mock works rather than they're testing [crosstalk]. BEN: Yeah. CHUCK: [Chuckles] Awesome. BEN: I've fallen into that, too, and then you're like, “What am I actually proving here? Nothing.” That’s part of the [inaudible] red-green refactor would be a good step to follow because you needed to see it fail and see that it’s going to pass once you write production code. But if you're doing all your work in your test and your test passes, that should tell you something. JAIM: I actually have that cases where I've done partial mocks on the code I'm testing; if you’ve inherited some massive viewController that does all these crazy things, everyone [inaudible] put into partial mock saying, ‘just don’t break or blow up if I call this.’ So yeah, it can be used in some cases, but in general, yeah, bad idea. PETE: It’s a good power too, but you need to understand it’s a bad idea in a lot of circumstances. JAIM: Definitely. CHUCK: I'm still waiting for the “with great power comes great responsibility.” PETE: [Chuckles] I just [crosstalk]. JAIM: It’s a framework for adults. PETE: [Laughs] Grownups only. I just noticed that the OCMock – Eric Dörnenburg looks like, it looks like he’s working on OCMock3, which does do Mockito style, verify after doing, so that’s interesting. JAIM: Oh, nice! PETE: Yeah, I'm not sure how long that’s been going. He actually – he’s a ThoughtWorker and he was posting on an internal mailing list about the 10-year anniversary and I think he’s got some renewed energy to brush some of the cobwebs off of OCMock since it’s been around for 10 years and it’s kind of a veteran at this point. CHUCK: Interesting. Should we get to the picks? You guys keep on saying cool stuff and so I will keep waiting. Jaim, you wanna start us off with picks? JAIM: Oh man. If I pick Mocks aren’t Stubs, am I taking our pick, Pete? PETE: You're very welcome to because I have another pick. JAIM: Okay. PETE: Yes you are, that’s okay. JAIM: We can share the [inaudible] now. CHUCK: Fight! Fight! Fight! JAIM: You can give it a +1. I was first, so I get to go first. But if you wanna get into pendactic arguments about what a stub is and what a mock is, you should probably read it because everyone talks about it differently, as I've learned today. BEN: What is a pendact? JAIM: Pandetic –. CHUCK: Pedantic. JAIM: It’s a word I can’t – pedantic, thank you. BEN: So if I'm being pedantic, I'm going to correct your pronunciation of that word. JAIM: Thank you, I appreciate the correction. Words today, I don’t know, they're not working for me. CHUCK: It’s a Minnesota thing – pedantic. JAIM: We’ve got our own pronunciation of some words. But anyway, that’s a great article to read to get a base overview of what we talked about and get an idea for how everyone talks about it differently. And I'm going to make a pick for the comedian Steven Wright, because I misapplied the Hedberg quote tonight. I feel bad about it, but much of my life, I used to annoy people – if they ask me if I slept well, I would say, “No, I made a couple of mistakes.” And that’s a Steven Wright joke, and he’s funny.  [inaudible] that I confused Mitch Hedberg with Steven Wright jokes, because Steven Wright – you don’t really hear much about him anymore, but great comedian. If you like Mitch Hedberg, you'll love Steven Wright. BEN: Cool, I have to check that out. CHUCK: Alright. Pete, what are your picks? PETE: I've got a couple. Well I've got a +1 on that Martin Fowler article. The other one, I think I might have picked this book before. Does this book called xUnit Patterns – it is encyclopedic in size; it’s very big. It’s a really, really good read. It also has a good website, xunitpatterns.com and if you go to xunitpatterns.com/testdouble, or if you just go to xunitpatterns.com and search for test double, it has a probably alternative definition of all these different things – test stub, test stubs, test spies, mock objects, fake objects, etcetera. The website has almost all of the patterns but kind of in reduced form. The book is definitely worth buying. I would be amazed if you managed to make it all the way through; it’s really big, but it’s a great reference material. It goes into this in a lot of detail, so that’s my first pick. My second pick is a shameless plug for a new thing that I've been partnering around with releasing for the longest time, so I'm just going to release it and see if people wanna use it. It’s a thing called postcards, and it’s for doing visual feedback from testers. The idea is you install this little thing in your little framework in your iOS app, and then when someone wants to report a bug or a visual defect would just give you feedback, they do this magical triple tap, and then they fill in a form, and then they hit submit, and then you get a screenshot of what they were doing in their postcard. So that’s at www.postcard.es. And I’ll have to get feedback from people on that, so use it and tell me what you think. CHUCK: Good deal. It sounds interesting. BEN: I have some instant feedback. Real time feedback. PETE: Okay. BEN: Dub-dub-dub is required. [Chuckles] We should do it with just the root. PETE: Yeah, I know. I'm hoping that by actually putting this out here, then by the time the podcast is released, I’ll be like, “Oh, I need to my little to-do list of 15 things that I've got to get done. BEN: That’s cool; that’s a good idea. CHUCK: Alright. Ben, what are your picks? BEN: One is an AFNetworking extension called AFHARchiver by a friend of the show, Kevin Harwood. HAR is HTTPARchive, and so AFHARchiver is a way to basically record and save off an archive of network requests that were made, so again, if someone’s testing your app and they're like, “No, I got this weird error” or “it crashed” or something like that, you can pull up the slog of all the requests that were made. That’s actually cool to ship in like a QA version of your app. I've also been enjoying this iPad game; it’s quite old, it’s called The Room. Right now it’s 99 cents and there's a sequel, so you have – it’s kinda like waiting for three seasons of shows to show up in Netflix so you can just watch them all at once. I really enjoyed this; it’s a puzzle game. I basically played it non-stop until I beat it, so I’ll be picking up The Room 2 pretty soon. That’s a lot of fun. Also to shameless self-promotion picks, one of them – I don’t think I mentioned on the show that I released GiggleTouch again, a big update, to GiggleTouch, which is an app for toddlers. Oh yeah, so go to GiggleTouch.com and there's a link to the App Store. It’s an update, so that if you already had it, you just get it for free. Otherwise, it’s 99 cents, but it’s a game I made with Sprite Kit. Also, I ordered some new shirts and hoodies for NSScreencast, and the reason I would mention that is that if you want to get an order in and have it shipped for you for US residents before WWDC, time is running out to get that out. So if you're interested in a shirt or sticker or something for NSScreencast. Go check out the store. And then lastly, a beer pick. Really, really been enjoying Dirt Wolf double IPA by Victory Brewing Company. Victory just makes really, really good beers in general and Dirt Wolf is no exception. If you see that, pick it up. CHUCK: Alright, well I'm going to jump in. I just have one pick, and that is, I was looking around for kind of a DIY, adjustable, standing desk. I want something that's electronic; I don’t wanna have to adjust it manually. I found one – I’ll put the link in the show notes – but basically it’s one that you can build just out of wood and you get some linear actuators, and it seems to work really well so I'm looking forward to building that. I guess I have to disclaim: I haven't tried it yet, but yeah, if you're interested then I’ll have a link to that in the show notes and you can check it out. Thanks for coming! [Hosting and bandwidth provided by the Blue Box Group. Check them out at BlueBox.net.] [Bandwidth for this segment is provided by CacheFly, the world’s fastest CDN. Deliver your content fast with CacheFly. Visit cachefly.com to learn more]

Sign up for the Newsletter

Join our newsletter and get updates in your inbox. We won’t spam you and we respect your privacy.