048 iPhreaks Show - Sprite Kit with Jonathan Penn
The panelists discuss Sprite Kit with Jonathan Penn.
[Bandwidth for this segment is provided by CacheFly, the world’s fastest CDN. Deliver your content fast with CacheFly. Visit cachefly.com to learn more] BEN: Hello everybody and welcome to episode 48 of the iPhreaks Show. This week on our panel we have Andrew Madsen. ANDREW: Hi, from Salt Lake City. BEN: Jaim Zuber. JAIM: Hello, from Minneapolis. BEN: And my name is Ben Scheirman from Houston, Texas and this week on our show, we have a special guest and that’s Jonathan Penn. So Jonathan, can you introduce yourself so that folks out there know who you are? JONATHAN: Yeah, I am known for doing a lot of iOS development. I write a blog called cocoamanifest.net, I have been running a company with my cohort, Josh Smith. We run Rubber City Wizards; we do mobile consulting and training and book writing, and we just started a blog called memorycrusher.com – trying to stay busy, you now. We are in the process of releasing a beta book to the Pragmatic Programmers about Sprite Kit – I'm pretty excited about that. BEN: Awesome, yeah. So Sprite Kit is the topic of the day. Just a little background, I guess, if you wanted to do 2D games a couple of years ago, Cocos2d was probably your best option. I spent a fair amount of time with Cocos2d; we did a game in it, I did a lot of tinkering with it. It seemed like Sprite Kit is like totally inspired by Cocos2d – I don’t know if you have any comments on that, just to get started. JONATHAN: Yeah, there's quite an uncanny resemblance in the APIs. From what I understand, there was someone who – a lot of this is just hearsay and criminology that we do on the outside of the Cupertino Castle, but from what I heard is that they hired some people who were involved, who at least contributed or have been part of the Cocos2d community to help them rethink how it would look to have a very well-integrated game development platform as part of the Cocoa world, and that’s how this was born. BEN: Yeah, so you have I guess the normal classes that you would sort of expect if you're doing a 2D game. You got like a scene and a scene graph with nodes, and you can do animations – I don’t know. Where shall we start? Maybe like a Hello World walkthrough in Sprite Kit? JONATHAN: Sure. Probably the biggest thing Josh and I have done some workshops with the Cocoa Conference tour, and as we’ve been teaching Sprite Kit, we’ve seen a lot of common questions asked by newcomers to the whole situation. The biggest one is, “What the heck is a scene graph?” For people who do a lot of iOS or Cocoa development, they're familiar with Apple’s certain flavor of Model View Controller, and the way that they, at least, tend to encourage us to structure our applications with a View and View Controller layer to help the display and transition things and some sort of Model layer that goes on behind the scenes that maybe your database and network layer – stuff like that. When people come to Sprite Kit and they see, “Wow, the ship!” if you have a spaceship game; the ship has its position, it also has damage values, and maybe its speed and stuff like this, like, “Where’s the Model? Where’s the View?” What we kinda help people understand is that with the scene graph, which is a different paradigm to organize your code and the cognitive process of writing 2D games – the screen graph represents everything. It’s kind of like the state of the game, as well as what the game engine uses to decide how to render things to the screen. And the scene graph is like you said it – you start with a scene at the bottom, and then you have these nodes, and nodes might have child nodes. Everything’s a node; even the scene’s at the very root of this whole graph, and that whole node tree helps the rendering engine how to lay everything out; it knows what's relative to what other things at different coordinate systems. You have a bunch of nodes that are all child nodes of a super node; you can apply transformations, scaling, rotation to that super node and everything inside it just rotates along with it, as if it was all its own subscene in certain ways. So all that together makes it really easy to structure a game so that you have different parts – the high score display, you got your enemies, you got your base, you got whatever – you're trying to structure the entire game using the scene graph as like a big tree. BEN: So in that sense, it’s not really unlike the view hierarchy in iOS? JONATHAN: No, not at all. It’s very, very similar to the view hierarchy. It’s just that in this case, the state of the game – it’s okay if the state of the game is wrapped up in the scene graph as well. A lot of people think, “Okay, should we pull the game logic out into a Model layer” – and this is not necessarily wrong, especially if you –. Say you have an artificial intelligence you're trying to do to make certain enemies move, you might want to make that reusable as a different NS objects; there's nothing necessarily wrong with that. But as far as encoding the state of the game at its current point in the scene graph is part of its charm, and part of the way that 2D games are envisioned as people code them. BEN: I think one of the things that attracts me to game development is it’s so different from your day-to-day forms over data, webpages or iOS apps, that it’s just a different way of style of software development that can totally stretch your notions about how software should work, and it definitely depends on the context, and game is certainly a way to just have a stark contrast of what you do day-to-day – unless you're a game developer. JONATHAN: Yeah, of course. BEN: So maybe we could talk a little bit about drawing – what types of things can you draw on the screen? JONATHAN: The simplest and most common way to draw is to use images as textures. The term they call them are ‘sprites.’ You have these special kind of sprite nodes that take images and then they become things that exist on the screen. You can move them around by changing their position; they have an intrinsic size; you can change the size if you want to make it look bigger or smaller, rotate them, things like that. But these are individual nodes in their own right; they just happen to be visually represented by a texture, and then you can use them as standalone or combine them in different ways on a scene. BEN: Is .png the best format to choose for this, or is there something more appropriate? JONATHAN: Apple has chosen to make .png the way to go. They already have a set of tools in Xcode to optimize .pngs for mobile devices, and the Sprite Kit engine is optimized to take .pngs and turn them into openGL textures under the hood to make all that magic work. BEN: Okay, interesting. Any particular type of .png – should you go for the 8-bit with alpha transparency if you can get away with it, or 24-bit is usually the norm? JONATHAN: It doesn’t really matter because Xcode’s .png cruncher – I think it’s actually what it’s called. There's a ‘build’ step when you build in iOS application. It actually takes an [inaudible] you tossed into Xcode as part of the bundle; it’ll take it and mash it into a completely different byte structure that’s optimized to load quickly on iOS devices, so it doesn’t really matter. BEN: Oh, okay. That's interesting. I'm familiar with Pngcrush, because there's a utility you can download from Apple as well. And if you ever crack open an IPA – not a beer, Jaim, but an actual IPA bundle. [Chuckles] If you crack that open, and there's a bunch of .png files that you actually can’t open and find there unless you un-crush them first. I used to use a utility – it was actually a little Ruby script that we go through in an IPA, like unzip payload and un-crush a bunch of .pngs, but nowadays I use the Slender app, which sort of does that a lot easier. But enough about that. Okay, so you mentioned that it gets converted to an openGL texture, so this is definitely going to be like more performant than throwing Views around and doing game development with UI views right? JONATHAN: Yes, yes. And I've done game development with UI views for some educational games, and it works up to a certain point. Because there's a lot of similarities – UI views, the whole subview hierarchy, each view has its own coordinate system. There's some actual benefits to doing a UI view-based game and every view also could have its own gesture recognizer, and some of the touch handling can be – Apple [inaudible] had to break that up for user interfaces. But with the way that UI views are drawn and the way the math all works, there's a lot more going on, so the more views you have, the slower the machine will get. You'll it the limits very quickly with UI views, and that’s where using things like Sprite Kit is just optimized to the hilt for shoving these textured Sprite nodes around as fast as possible, with the least amount of friction to the system, so that you can get a whole variety of frenzy happening on the screen to delight the player and all that stuff. BEN: Do you have to exhibit the same sort of caution when doing layer transparency? Pretty common, you're not going to have a perfect rectangle sprite; you're going to have some sort of transparency in your sprites, and it’s going to have to blend that with background. And I know openGL makes that – it’s optimized for that, you know, to composite things together. I'm just wondering if there's similar considerations we have to make when doing transparency on our sprites. JONATHAN: There are some. I've been able to tax Sprite Kit with the more transparent pixels you have in a texture into a .png image that you use. It does take more work to blend that with the things behind. You can tax Sprite Kit with more transparent textures or not, but it takes a lot more to cause a problem than it does in UI Kit. I mean, it’s built to be – you can throw a lot at it. You don’t have to worry about that too much, unless you have huge textures. There are ways of constructing your game where you don’t have as many sprite nodes there as you try to remove things that you don’t need when they're off the screen, Sprite Kit also has its own optimizations to try not to draw things that it can tell aren’t visible either, but there are hints that you could try to do to make that a little better. But for the most part, I mean, you don’t really have to think about optimization until you start reaching a problem. Or if you know, as you start the game, that you're going to have 10,000 nodes to represent what you're trying to do, you might want to re-think things. But for a lot of people experimenting, don’t worry about it; it’s not a big deal. BEN: I mean, there seems like a lot – like, if you have a spaceship game and you’ve got your spaceship player at the bottom, and you’ve got 12 or so spaceships at the top, kind of like Space Invaders, and then you're shooting bullets – I mean, we’re still under 100 nodes. But if you start adding in particles into the mix, I'm wondering if it’s easy to get above that 10,000 number you just tossed out, with things like particles. JONATHAN: The good news about particles is the way you can implement your own “particles” – really, all they are are just their own, individual nodes that travel like particles for an explosion, or smoke, or something like that. But Apple has a special, optimized particle emitter that uses an even lighter-weight type of node that you can create with the APIs Apple’s provided, and you can get 1,000 particles on the screen, with even less of a performance hit than if you were to draw 1,000 sprite nodes yourself. I mean, if you're using Apple’s particle emitter, don’t worry about it. You're not going to hit much of a ceiling at all. But there is a caveat: ironically, even though when we use the simulator to do the iOS development, we’re usually told to be careful because the simulator runs faster on the Mac than the devices, so you have to do performance testing on the device because it might be slower. Ironically –. BEN: Right, [inaudible] talking about drastically different hardware there. JONATHAN: Yes, but the funny thing is, the rendering pipeline on iOS for openGL is so well-tuned that the emulation of openGL [inaudible] in the simulator is slower than the device, so if you start experiencing slowdowns, don’t give up yet. You should be testing it on the oldest device you plan to support just to see it’s possible that you're not even causing a hiccup at all on the device itself, because there's not an emulation layer like you have on the simulator. JAIM: And we should point out that if you use the Sprite Kit sample that it shows either frames per second counter on there, which you should pay attention to for, I guess, throughout development, just to make sure that you're meeting your target frame rate. JONATHAN: Yeah, yeah. And that frame rate is kinda like a guide – it’ll dip up or down, depending on different things that you're doing. Really use that as a way of kinda taking the temperature of your game. It’ll tell you too how many nodes you have on the screen – if you’ve got 600 nodes, then your frame rate drops to 50 fps for a few seconds here or there, may not be that bad, but if drops consistently to 40, 30 fps, then you might want to rethink how you're calculating things and maybe batch operations and stuff like that. BEN: I assumed that Sprite Kit gives you a nice game loop that you can run calculations like physics, or a collision detection, or am I out of lives, do I need to [inaudible] over, things like that? JONATHAN: Yeah, the update loop is great. It’s also for people who are used to the normal event polling, or the even notification mechanism of standard UI, is the event loop can be kinda confusing to newcomers. You literally get a special method that’s called on your scene every frame, and you can do all sorts of magic in there, like move things around, check for – like you said – check for collisions or player death, player success, whatever, and do the work you need to do to help the game keep going. ANDREW: I wonder how checking for collisions, which you mentioned, works. I mean, do you have to go through your entire scene and look at every object and do some test on geometry, or is there some help to that in Sprite Kit? JONATHAN: That’s a great question, and it depends on your goals. At the simplest level, you can ask a node to give you its calculated frame, which is a rectangular, the rectangular region that it takes up including all of its subnodes – everything that's taken inside that node that you see. And then you can do some simple [inaudible] intersection to check for things. If you know that you only have enemies and then there's your player, so you would just loop over your enemies and check for intersections in some ways like that. It doesn’t always work because – it all depends on your goals. Doing pixel-perfect collision detection is a very hard problem. Sprite Kit doesn’t have that built in; at that level, you kinda have to decide what's close enough to make this a problem. But there's also a whole different mechanism of collision detection if you use the physics engine that’s built-in. This is new for Cocos2d people, because they're used to either having to pick box2d or chipmunk, and then live with their decision for the rest of their game, and then try to figure out how to integrate it into their system. With Sprite Kit, there's a physics world already on your scene; you don’t create anything – you just create physics bodies, attach them to your nodes with a certain –. The bodies have a certain boundary, whether it’s a circle or rectangle or whatever, and then you can have the physics engine tell you when these bodies touch. That’s a different class of situation; like in some cases that’s really useful, but it also means that you are now writing a physics game, which is not every game is a physics game, and you don’t have to get into that – it all depends on your goals. But that can be really useful too, and in that way, you get nice edge detection out of the box; you just have to now deal with the whole physics body world and worry about [inaudible] to decide which bodies are allowed to overlap or intersect, and which bodies should respond to each other. You then also have to deal with when they bump into each other – the physics engine tries to oblige and says, “Oh, I should make them bounce” or whatever –. You may not want that either; it depends on your game mechanic. But if you are doing a physics game, use the physics engine for collisions and you're fine; it'll just work. BEN: Is this a Sprite Kit physics engine, or is this something like UI Kit dynamics? JONATHAN: It’s not UI Kit dynamics; it’s its own thing welded into Sprite Kit. Although the technorati have dug into this system and found out that both UI Kit dynamics and Sprite Kit are linking in box2d, at least at the current time, so they share that common heritage, but they are very different systems. The UI Kit dynamics system isn’t really – it’s not a physics engine in a classical sense, because you don’t have access to a lot of things that you only have rectangular bodies, but with Sprite Kit’s physics engine, it feels more like what you'd see if you were used to using box before. There are some things that are kinda hidden away that you don’t get a chance to touch, but for the most part, you'll be very familiar with it if you’ve done other physics engine work. BEN: Cool. So I guess we covered a little bit about drawing and the update loop. What about user input? You mentioned that with UI views you get tap gesture recognizers and things; I'm guessing that we don’t have UI gesture recognizers on Sprite Kit. JONATHAN: Nope. I understand why, because the whole responder chain and then detecting hits within subviews is part of why UI views have a lot of weight to them, that makes them heavier, why you want to maybe not consider them for a game, but man, it’s nice. I do miss having gesture recognizers. Everything in Sprite Kit lives inside a UI view, and you can have a gesture recognizer on that UI view, and then respond to it. You have to delegate what the gesture recognizer tells you down to your scene, like literally. You’d register that you received the gesture recognizer events, and then you say to the scene, “Hey, you go tapped here.” And then the scene would respond to that. So it’s possible to use them, but you can’t attach them to individual nodes; you have to go look up nodes that are around where the finger is and then decide what to do with them. And that’s the same situation it was with Cocos. At least they could have every node have its own touchIsBegan methods, so there was lower level touch handling. In Sprite Kit, they chose to say that only the scene can receive raw touch events, like touchesBegan, moved and ended and then if you want your node to do something, you have to establish some API with that node, that you then call and say, “Oh yeah, you are tapped. Blow up” or whatever it’s supposed to do. It’s a little bit of a bummer, but in some ways, you know that your game engine’s [inaudible] BEN: It’s sort of acting almost like a controller? You know, that your input is handled at one place and then passed on. I could see that it could potentially be a mess if you had input handling just sort of federated out of a bunch of objects – I don’t know. And a lot of times the input depends on other nodes in the scene, right? So, one node may not know all about its sibling nodes. JONATHAN: Yeah, in many ways the scene is like the traffic cop for real that receives information about all the other nodes and then makes decisions about what should happen. I've constructed things where I had nodes know all about each other, or I might write methods on them to say, “Hey, this thing is here nearby. Do something different.” But for the most part, the scene is really the one directing everything. Passing touches along, interpreting events, reading the state of other nodes to decide what to do, and you kinda have to factor your game out from there. BEN: And so, since the scene is the one that handles touchesBegan, touchesEnded, typically in a UI Kit application, you would do something like touchLocationInView and you pass maybe self.view if you wanted the location in your outer view, but you could also pass in a location inside a different view and it will give you whether or not that point was inside that view. Is this something that you can do similar with nodes? JONATHAN: Right, yeah, and that happened because each subview has its own reference point, its own origin, local coordinate system. So the good news is, that works out of the box with Sprite Kit, too. Apple added a locationInNode method to UI Touch and you pass at any node, whether it’s the scene or any tiny sub-node farther down the tree, and then it will tell you to touch its coordinates within that local node’s coordinate space. The one caveat – and thankfully this method handles this – but the thing to remember is that for some reason, Sprite Kit’s coordinate system is flipped. It’s just like MacOS and it’s like Cocos2d as well. It’s not like UI Kit’s coordinate system where the origin is in the top left. In Sprite Kit, the origin is in the bottom left of the screen, and so you'd have to do that conversion yourself with just recognizers, or if you had the touch objects, you could say locationInNode, and it will have done that flipped conversion for you so everything lines up just fine. BEN: Yeah, I've definitely done stuff like that where I end up using code graphics to draw something into an image and then I realize that the whole thing is flipped. JONATHAN: Haven't we all? BEN: And then you just spply transform and forget that you ever did that. So yeah, I could see that that could be confusing at first. Okay so, let’s say you have a scene and then the scene you have, let’s say, you spawn 100 balloons and they're sort of bouncing across the edges of the screen and bouncing off of each other, and when you pop on a balloon, it pops. Would you have to, in the touchesBegan – or maybe touchesEnded, depending on what you actually wanted – in the touchesBegan, would you just loop over every single balloon testing to see if you had a hit test in that balloon, or is there some sort of facility to tell you like which nodes are under my finger right now? JONATHAN: Yeah, and unfortunately, there's just no way to get around the looping part, unless you took efforts to try to cache or maybe – there's different algorithms to segment the screen and then sort nodes by that and keep them on hand, you know you're in this quadrant. BEN: It’s like a data structure problem that [crosstalk]. JONATHAN: It totally is, yeah. With all the big, old problems, you have to try to solve that too. There is a method on nodes to basically, or on a scene, to say, “Hey, find me the leafest – if that’s even an adjective – the leafest node, find me the leafest node at this point,” but I bet money under the hood that it’s just looping too. If you know you have balloons – let’s say you’ve got 50, or you said 100 balloons – on the screen, you can name them all in a certain way, and then there's a method that lets you just say, “Hey, every node that starts with the name balloon, let me loop over that.” It’s an enumeration method. I'm assuming – I don’t know right now what it does – but I'm assuming Apple wants to optimize that looping and might even do it in a concurrent fashion, so that it tries to find all the balloons and then you're just looping over it and applying updates to the property [inaudible] seem to change the state of the game, move things around, explode them, whatever. I mean, there's ways, but yes – I mean, you have an array of 100 things and you have to find one of them, and so, that’s what computers do, they loop. BEN: Unfortunately. I love problems like that. I don’t know, maybe it’s a certain class of people that love data structures, but that sort of stuff fascinates me. I wouldn’t say I'm particularly great at them, but the problem and the solutions that people come up with are fascinating. I once read a book called Game Coding Complete by Mike McShaffry, and it’s a fascinating book. It’s really, really dated; covers direct3D, but he worked on the Thief Game for the PC, and one of the mechanisms of that game were that you had to stay in shadows, and that if you knocked over a bottle, people nearby would hear it. They designed their scene graph around the fact of what could hear things and at what amplitude or magnitude, which is really interesting. And so they designed their object model around that fact so when I make a sound, that it can sort of alert things that are in a radius around that sound, accounting for walls and things like that. Really, really interesting stuff. JONATHAN: Yeah, I mean, there's also optimizations you can do. And I haven't, thankfully, nothing I've written has been so complicated that looping over nodes has needed something real robust. BEN: We talked about the desired [inaudible] 60 fps. Do the math, that’s 16.7 ms in your update loop, so if you're looping around 100 nodes, you're fine. But if you're looping over 10,000 then, I don’t know, you gotta measure it, but you don’t have a whole lot of time to do your work in that method. JONATHAN: And I would bet that if you had 10,000 things on the screen that you are trying to loop over, you're also going to run into other problems. I bet you'd have figured out how to not have 10,000 things on the screen at the same time before you’ve reached the point where looping might have been –. Yeah, [crosstalk] BEN: Your trying that many things might cause problems before trying to update that many things. JONATHAN: Possibly, depending on how large they are, and what you're trying to do, yeah, it could get messy. JAIM: So are there real time considerations, or do you have all the time that you take for getting a cycle to do stuff in? JONATHAN: Well, in the update method that you're called, the only parameter you're passed is the time, the current time. There's like an ever- incrementing clock that’s guaranteed to increment and it’s based on the number of seconds that you're currently at. You can use that to check if you saved the last time that you recalled, you can use that to figure out how many milliseconds are between each frame, which helps a lot with doing calculations like moving things and all sorts of stuff. You really have to do a lot of fast work in that method, because if you – what was the value you said, Ben? BEN: 16.7 JONATHAN: Yeah. If you take longer than that, you will drop your frame rate, but it depends on how often you do that. The player won’t notice, unless you start dropping really low a lot. And we’re forgiving – we see frame rates drops all the time, and most of the time we don’t notice – unless you're a game maven or a performance nut. The real problem happens if you have to loop over, like in this example we had – 10,000 nodes every frame. You'd want to invest some time into an algorithmic solution that could slice that down somehow. BEN: Yeah, that thing you just mentioned about calculating the lapsed time is just common in just about every single video game development framework out there, because those of us old enough to remember the 286 and 386 PCs with the turbo button, and you go load up a game that was meant to play in 286 and all of a sudden the whole thing just runs way faster, because [crosstalk] JONATHAN: Ultra hard mode. BEN: I remember trying to play Commander Keen and Secret Agent Man and all those games way back when, and they just were not clock cycle independent – if that’s the term. They were like, “Oh yeah, this runs at 286 and it’s clock speed of x” and so they just base all of their animations and timing off of that instead of the amount of time since the last frame. JAIM: Seemed like the thing you'd do at that time. JONATHAN: It wasn’t a bad decision. I mean, yeah –. BEN: I mean literally, you have this turbo button on your machine, and literally you're like, “why would I ever un-check that?” [Crosstalk] Oh, I'm playing an old game; I’ll uncheck it so I could actually play it. JONATHAN: Nowadays, you can’t get away with that. Everything your game runs on is so variable that you just have to, you have to know – you have to have a clock. A real clock. BEN: Kids these days, they just don’t –. JAIM: So let’s talk a little bit about, how does this integrate with the rest of your app? You say that these scenes are kinda rendered inside a UI view? Do you typically run a whole Sprite Kit app by itself, or do you integrate it with kind of a standard UI view-type app? JONATHAN: A lot of people I've seen have used Sprite Kit thus far. It’s still a relatively new technology, and I am not aware of a game released in the App Store that uses it exclusively yet – I've not heard of one. But, I know a lot of people who are using this and they're dabbling in it, and thus far they’ve been building complete – they’ve been building the whole application as if it was all in Sprite Kit. All of it – like the ViewController, all it does is boot up your Sprite Kit view and then go. Everything happens to the side, with the menus and stuff like that. But because it’s part of the view hierarchy, you can easily have UI views and labels and buttons on top of your sprite view and have your ViewController respond to them. Tell your scene, “Oh, it’s time to transition to a new screen, it’s time to play the game.” Use the navigation controller hierarchy if you want. Its integration with UI Kit is really nice; the only bummer is that you can’t put a Sprite Kit view on top of UI views. It will never be transparent, because it’s like a raw openGL context, but you can lay anything else you want on top of it from UI Kit [inaudible]. BEN: That’s really good, because I can’t even tell you how many games I've used where they implemented their own cover flows, scrollview thing, or their own tableView and just – they're never going to get the physics right, the exact behavior that we’ve come to like. It’s like the test of all these HTML5 apps that are out there who are, you look at them and they're like, “No, I can tell it’s not real. It’s not using scrollview.” There's some pretty good fakes out there like Letterpress is probably the most convincing, but yeah, people notice those things. JONATHAN: On that note, I would recommend checking out the UI scrollview session from WWDC 2013 because they have an awesome example of how you could use a scrollview that doesn’t in itself contain any content – you're just using it as an interface mechanism. And then you read the values of the current scrollview to then [inaudible] manipulate something else. They tie it to an openGL rotating cube, but you could easily use a scrollview, a content-less scrollview, in a Sprite Kit game that way, and then the physics just translate because the content offset values you get, you'll just use them however you need [inaudible]. It works. BEN: That session was really good. One of the engineers on scrollviews – Josh Shaffer – anyway, he was talking to Loren Brichter about Letterpress, and he was like, “Hey, you almost fooled me.” I almost thought it was scrollview and then he realized that he couldn’t walk the scrollview with his fingers. Other than that, it was a pretty darn close match. JONATHAN: Man, those little details. JAIM: Just getting into some other games, I've been playing a lot of Zelda for the 3DS. And a game like that, if you just think of the 2D SuperNintendo Zelda, or even the Nintendo version, you'd probably build that with a tile map, and then you have your level and when you reach the edge, you would sort of just transition over to the next sort of viewport on the overall world. How might you build something like that with Sprite Kit? Or are there built-in support for things like tile maps? JONATHAN: Alas, not yet. But given that Apple has made – like a lot about Sprite Kit is built to be serialized, even the Sprite Kit, the emitter nodes, the editor in Xcode that lets you edit particle emissions is basically just writing an archived SK emitter node to disk that you load with nskey to an archiver. So that the idea that your entire scene could be serialized this way is – I have no doubt that Apple has plans for other tools that they're going to put into Xcode as well. But we could easily build our own and then just serialize these things to disk and they work fine. And several Cocos2d tilemap tools are now declaring Sprite Kit support, so it’s not that hard to use the same mechanism. BEN: Yeah, that really was what I was referring to, is that there's a lot of great support for – I mean, I guess people aren’t familiar with the tilemaps. It’s like you have an editor separate from your game and you have a designer come up with your various pieces. So if I was doing Zelda and have like the yellow sand tile, and I'd have the red rock tile, and I'd have a push tile, and maybe a door tile, whatever, and they have 50 of those or something – you can get by with a lot less. And then literally, it’s like MS Paint – you just click on one of them and that’s your brush, and you start painting on physical tiles. You just divide up your scene in equal chunks and just fill it in, and then that becomes data that your game then reads and knows how to render. It’s fun to build things like that, but if you're trying to build a game, it’s nicer when your engine can do it for you so that you can actually focus on building a game rather than building an engine. JONATHAN: You don’t know how you want the level to work until you kinda get a feel for it, and being able to see it really helps. ANDREW: I dabbled a little bit with Cocos2d really just a tiny bit, but I know some of those tools also – so you have all the frames from s sprite and they pack them into a sprite sheet, and that’s some sort of optimization, those are faster to load or less memory. Is that applicable to Sprite Kit and can you explain that a little if it is? JONATHAN: Yeah, so the idea is by default, if you just threw .png files into your bundle, when your sprite nodes load them in, each one is a separate openGL texture, which is fine – it’s very fast in its own right because of openGL. But at some point, when you have hundreds of these going on, you're now shoveling around 100 textures. There's a memory and performance hit with that. So the idea is, if we had one texture that was big, but had each of these individual sprites on the texture, and then when I had a sprite node of my ball, and then a sprite node of my bat, they would both be referencing the same texture but just different coordinates within that texture in like a different window so that they would have their different images. That’s a lot faster, because then openGL just loves that under the hood, and the beautiful thing about Sprite Kit is it can build that texture atlas for you automatically without any effort on your part. You create a special folder with the name ‘atlas’ at the end, toss it into your Xcode project, and just throw files in there. You don’t change a line of code; you don’t do anything different. If it sees a texture atlas with a name that’s in that directory, and then it knows what to do, Xcode automatically packs it at the densest format it can while making it the most performant. It’s actually one of the coolest integration points, when I show people how that – “Oh yeah, here, you just throw your images in here. What else do you have to do?” Nothing. Your game is just faster. It’s pretty cool to see that work. ANDREW: Okay, that is cool. I know there are third party tools that do that with Cocos2d and it was actually something that kinda confused me when I was trying to get into that stuff. BEN: You can actually pack these things quite condensed, but the getting them out at the right levels is kind of a pain in the ass. JONATHAN: If you have a larger – I think it’s 2048x2048 – texture, you have to have now two textures. You have to split them and Xcode does all that automatically too. You never see the difference – it just works. BEN: I once gave a talk on Cocos2d in Vancouver; I’ll put a link in the show notes to the Super Metroid Samus Sprite sheet, and it shows basically every animation frame for the Metroid game. You got Samus running right, running left, jumping, running, aiming the gun at various directions. So I did basically just the running and the jumping animations, and looking at the sprite sheet, it looks like it’s laid out pretty much like on a grid, but each frame is variable size. Because I didn’t have anything that I built, I didn’t have the actual points and so literally it was just guesswork. Like, “Okay, one pixel over,” “Oh, that’s too far. Let me change my whole window” because I was trying to deal with frames of specific size. It was definitely a hair pulling out experience, but the end result was pretty darn awesome. I created an endless runner where your character was Samus. JONATHAN: They didn’t share the points?! That’s like, cool! BEN: I think someone extracted it out of the Metroid ROM or something. JONATHAN: Oh, okay. BEN: So, I don’t know. I mean, it’s thievery, so. JONATHAN: Touché. BEN: Yeah, this stuff is pretty cool. How do you do all – like once you have all of your sprites on sprite sheet, do you create a special type of SK node that knows about each individual frame? And if so, how do you sort of advance [inaudible] tell them what animation you're running? Because like on that sprite sheet I just linked to, there's lots of sets of animations. You’ve got the running one, and there's like 12 frames, and you get the jumping one where there's five frames or something. JONATHAN: In the Sprite Kit world, you wouldn’t take a preexisting texture map like that, or texture atlas. You would name all your files in a way that you could remember them, same as run1, same as run2, same as duck1, same as duck2. And then you would then use those as the names – that’s how you identify them inside your application, and you wouldn’t care that they're in a texture map or not. You'd just refer to them as if they were individual images and Sprite Kit sorts it all out. But you would just have a simple, it’s a sprite node, there's nothing fancy about it. There's an action that you'd run on the node to play a series of these textures, and that gives you the frame-based animation. You have a duration between each frame, and then you can say ‘repeat this forever’ and then the action that you run on this node makes it go. When it’s time to stop running, you would stop that action and then run a new one, like maybe same as resting one, two and three, stuff like that. It’s funny, now that you bring this up, I don’t think you could just take this texture map as is and then figure out how to make it work with SK’s sprite node, because you'd have to reverse-engineer the points and then figure out how to make Sprite Kit do it. It would just be easy to slice everything up as individual files –. BEN: And let Xcode do it. JONATHAN: Exactly. You shouldn’t think about texture atlases; you should let Xcode do that. It’s just easier. BEN: So the other aspect of this is that the frames were variable-sized rectangles. I imagine if you wanted to do this with Xcode you probably want to make them all the same size and sort of center the character. Because what would end up happening is when I had it wrong, the character would sort of be running in place, but then every frame would then jump to the right. Every time I got to that frame, it’d jump to the right a couple of pixels and I’ll have to adjust it when it’d jump to the right – only one pixel, then I just do it again. Tedious work. JONATHAN: Yeah, that would get complicated. There probably are different rules, like if you have a physics body, if this was part of a physics game and you had a Samus physics body, you'd have to adjust its size too to match, you have a little ball shape as opposed to the standing person [inaudible]. This is why game development is hard – there's just a lot of things you have to think through to make sure that they all work right. BEN: I wanna post a link to this game, just so people can take a look at it. There's a screenshot [inaudible] as well. JONATHAN: Rock on. BEN: It’s not really a game, it’s more of just a demo of doing stuff with Cocos2d. An interesting point about this is that it’s an endless runner, so the idea is you got a parallax background, you're running, and there are no enemies or obstacles because I only had to do this in an hour. Basically just running on a surface and jumping. And so since it’s an endless runner, really the player is just stationary and the scenery move by. And so I had like a ground texture that was wider than the screen, and I would just sort of move it across the screen, and I would have two of them sort of side by side. And when one of them was completely off-screen, I would just throw it away and create a new one that would be just beyond the one that is onscreen. Right? So it’s sort of it’s creating a ground on your feet. And what ended up happening is that you run into the allocating objects and [inaudible] new objects is noticeable inside of a loop like that and so I ended up taking a play out of the UI tableview playbook and just using cellReuse. And so I just used the same ground texture as before and put that on [inaudible] and it was funny that that wasn’t obvious to me at first, but then when you run the game it would pick up every so often and moving that made it a lot smoother. JONATHAN: Yeah, that works really well. It’s funny you’ve mentioned that; for some reason, creating SK node objects is really fast; you shouldn’t be afraid of creating new ones. Reusing them is a good idea, of course, but in my experience I've been able to have creating hundreds of nodes at once and not nearly a peak out of the machine; it just hummed along fine. BEN: I think, maybe, the problem was related to the fact that this was attached to the image, so maybe the texture loading was causing the problem, not the alloc, but –. JONATHAN: Oh, like it wasn’t caching the texture for you or something like that? BEN: Yeah, maybe. JONATHAN: Okay, gotcha. ANDREW: I think the last thing that we haven't talked about that I was curious about is sound in Sprite Kit. Games all have sound effects, and I have no idea how that's handled and I'm curious about it. JONATHAN: At the simplest, there's an action – and we didn’t really talk much about actions, but there's just these objects that you can run on any node in the scene, even the scene itself, and they just – they do things, they move nodes, they rotate them, whatever – and there's an action called runSoundFileName that will just play a sound file. Any sound file you throw at it, whatever the low-level sound APIs can handle – .m4a, .mp3, .aif. From what I've read – I've read conflicting information about this in Apple stocks – but from what I've read, it sounds like the .caf format, Apple’s Core Audio Format, is like the equivalent of Apple’s crunched .png for images, but this is for sound. The most performant way of loading it into memory and using it, but it doesn’t really matter. I use .m4as in things that I do – the .aac-encoded audio files – and you're going to want to pre-load them at the start of the game anyway so they're already on memory, and when you go to play them, they're already decoded. But it doesn’t really matter what you do at that point. Unless you have lot of sounds, and at that point, the .caf format is larger than .mp3 or .m4a because of the way that it’s encoded. It’s also not a lossless format or not a lossy format, so if you're worried about your bundle size, you probably wanna use smaller sound files or not, depending on how you're doing this stuff. But that is for one shot or simple triggered sound effects. If you want background music, just reach for the AV foundation core audio APIs. AV foundation actually works really well [inaudible] audio players, an excellent class that does looping; you can control the volume on it, you can turn it up and down, you manipulate it in any way you want, and it’s just an objective-C object, so there's no dangerous C-bits to poke yourself on. That works fine, I mean, if you wanna get more advanced, you could probably reach for open AL. I know that Cocos – I forgot the name of it – Cocos had a wrapper of some kind around open AL for 3D audio and stuff. I think it was called Denshion, or Denison? ANDREW: Oh yeah, I think that rings a bell. JONATHAN: I hadn’t messed with it, and none of my audio needs have ever been beyond stereo panning, so it’s not been that bad. But yea, audio is – because of that runSoundFile action, it’s even simpler to play audio than it is elsewhere on iOS. But otherwise, just reach for the stuff that iOS has and you'll be fine. It makes this audio just like you expect; if you're playing an audio file with AV audio player, and then you trigger one-shot sound effects – you have complete control over how it all works. ANDREW: Okay, it sounds like it’s pretty simple. That is called CocosDenshion; I don’t know that Denshion means but –. BEN: Yeah, it’s kind of a weird name, but – [inaudible] Google auto-suggest helped. JONATHAN: Wow. [Inaudible] BEN: You know how you, if you animate UI views, there is – and really we’re talking about layers at this point where you have a position of a layer, and then you have the position of the intermediary position like during an animation. So if you were to do the animation of two UI views and continually inspect the frame of the view, it will just always be at the ending frame. Like, you wouldn’t see the interpolated positions unless you dive into CA layer and forget what the property is you can observe for that. JONATHAN: Yeah, level model, animation model versus the animation reality – whatever [inaudible]. BEN: Something like that, yeah. I mean, there is a way to get actual pixel position of a layer, but with UI view frames, you don’t get that. Is that similar in Sprite Kit nodes, when you run an action and say, “Okay, move over here. Rotate. Move over here, rotate. Move over there, rotate” and then I want to have like a turret just like continually pointing at the character? Then in my update link, I would need to say, “Okay, where is the character? What angle is he form me?” You know what I'm saying, right? JONATHAN: Yup. BEN: So in your runAction, I would still – like in my update – would give the actual pixel positions of each player. Is that just how it works, or –? JONATHAN: Yeah. BEN: Okay. JONATHAN: And I believe it was the same way in Cocos. Well, I guess with Cocos, with actions in Cocos it works the same way. With the physics engine, if you were running box2d, you had to go update the position of your objects based on boxes current state of the physics [inaudible]. All that’s handled automatically with Sprite Kit and whether you're running an action to move a node, or whether you're using the physics bodies – everything is, at any point in time that you access a node’s position or its angle, it will be what the player sees at that point. Or, give or take, depending on if you accessed it before or after the frame drawn. But the idea is that there is no – like in core animation – there is no intermediate model of the animation that’s being run independent of the beginning or ending state. It’s all – the properties on the nodes are being manipulated in real time by the actions in the physics world, and you can query, and then change things on the fly when you're ready. We need to get rid of the falling velocity on this object because it needs to fall up or something like that. Once you change things, everything stops on the [inaudible] and moves. BEN: So any other last questions we should cover that you guys have from building games in Sprite Kit? If you're doing this at home, pull out your trigonometry book, because it’ll come in handy. JONATHAN: Oh my gosh, yeah. We’ve had a lot of questions about that in our workshops, and yeah, we’re thinking about possibly writing a math book for game developers. There's a couple of books out there, which is a great physics book for game developers by O’Reilly, but a lot of them – yeah, we’re not sure yet how to do this. Because a lot of people come with different experiences, different memories, you know? How long ago did you take trigonometry and how useful has it been in your career thus far? BEN: How cool was it when you took it? JONATHAN: Right. BEN: If you're into game development at the time you took trigonometry, it’s probably your favorite subject. JONATHAN: Or if there was someone you liked who happened to also be in the class, you know. BEN: [Chuckles] JONATHAN: Social issues –. I guess a lot of this – yeah, we’re trying to work out a way of helping people wrap their minds around it, because in some way it’s kind of like an inverse word problem. I hated word problems growing up because they felt so disconnected from reality, but in this case you're like, “I want to point this tank turret to this angle where the player is. How do I find that angle?” And once you see it done, you're like, “Oh, yeah” you kinda understand it, you might even start remembering the SOH-CAH-TOAH phrase and the hypotenuses, adjacents, opposites [inaudible] back in your head. But recognizing that pattern – I have a point and this forms a right triangle, and here’s how I can calculate the angle – seeing those patterns as you look at the game you're trying to create, I think there's probably a place to help people see through that, so, we’re talking about it. I’ll put it this way – I have not found a book that has satisfied me given that criteria. BEN: I think it would be good; I would certainly read a book like that because I am in the camp that thought it was interesting, I was decent at it at that time and then forgot a lot of it, so I know that there's probably an easy math solution to some of the problems that I wanna solve, but it’s not on the tip of my tongue, so [inaudible] to go and look it up. The internet is a great resource for this sort of stuff because people are generally pretty friendly and if you can find a math formula for it, you can model it in code, and that's pretty easy. I don’t know; it’s pretty interesting stuff. I like the applied math and physics. Trigonometry, for instance, and Physics I in college were super, super entertaining for me because Physics I – it’s like [inaudible]. Remember the – what's it called? Gorillas.bas on MS-DOS – do you guys remember that? No? [Crosstalk] The gorillas game where you – you probably played 10 games like this. It’s gorillas on buildings and you have to throw a banana to hit the other gorilla. JONATHAN: [inaudible] BEN: The angle and the velocity, it throws the banana and then like literally, just sort of takes the chunk out of any buildings that you happen to hit along the way. And the first person to hit the other person accurately – and there's wind and stuff like that. But I mean, this is exactly the Physics I class in a nutshell, so that's why that stuff was fascinating to me. So pay attention to school, kids! [Chuckles] JONATHAN: Now I have a reason for my kids to listen to this podcast. Good stuff. BEN: Yeah, I mean, I don’t know the average age of this podcast – probably not in the teens, but who knows? JONATHAN: Don’t sell yourself short. [Chuckles] BEN: Yeah. Okay, well I guess we’re probably about at the end of the questions, so let’s go ahead and go into the picks. Jaim, you wanna start us off? JAIM: Sure! So, hey kids! Do you like to rock ‘n roll? I like to rock ‘n roll, and a great track [inaudible] has been headed down our generation. Have you guys heard Stooges: Raw Power? Basically the CD version we can get most of our adult lives has been unlistenable. Iggy pop remastered it in the ‘90s and it was basically unlistenable. [Inaudible] lousy [inaudible] it sounds very punk rock, but the remastering was just awful on it. I think it has the record of being the loudest CD ever made. But as I was shopping a few months ago, I noticed they re-released the original David Bowie mix for The Stooges: Raw Power, and it’s much more listenable. I had the old Iggy pop mix for a long time and I rocked out to Search and Destroy, and before Grand Theft Auto, this is what you did to get crazy in a car. You just crank up Search and Destroy, but we didn’t have that back then. BEN: I think you just alienated our teenage audience. [Chuckling] JAIM: In fact, no, no. I think, here’s how to rock and roll and anger your parents. So I'm bringing it up because Steve Ashton from the Stooges, he passed away yesterday. He came up on the Twitterverse, so, sad day but find yourself a copy of Raw Power, the David Bowie mix, and don’t drive if you don’t wanna crash into stuff. That’s my pick. BEN: Okay. [Chuckles] Andrew, you wanna go next? ANDREW: Sure. I have three picks today, and my first one is actually a Ray Wenderlich tutorial. Ray Wenderlich’s got a great site full of good tutorials, but I actually don’t read it a lot. They posted a tutorial about C++ for iOS developers this week, and I'm really kind of a novice C++ programmer, and this was just a good – it’s part one of a series that they're going to do more parts in, but it was just a good overview of C++ coming from a perspective of somebody who’s very comfortable with objective-C, and it made it seem not so hard although of course they didn’t get into any of the really sharp edges in C++. And then my second pick is the RFduino. This is a really tiny, little Arduino that’s got a built-in Bluetooth 4.0 transceiver, so makes it super easy to create a little Bluetooth 4.0-enabled piece of hardware and of course you can talk to that with your iOS device, and there are some really cool stuff that you can do with Bluetooth 4.0 in core Bluetooth and this makes it easy and they're like $20 – pretty cheap. And then my last pick, inspired by a mention from you, Ben, in the show is Commander Keen. This is a game from the very early ‘90s that is a 2D sidescroller and it’s got kind of an interesting history, I think. Originally, what happened is the developers wrote a Super Mario Bros clone, and they took it to Nintendo and said, “We made this version of Super Mario Bros for the PC, do you guys wanna basically license it from us and release it?” and Nintendo said, “No, we don’t wanna do computer games. We have our consoles” so they just sort of reworked it really and used the engine that they had written for that to make their own game and it was really successful. And it’s actually available on Steam – episodes one through five – six is not there for some reason, but one through five are on Steam and you can buy them for $5 and play them on your modern computer. So those are my picks. BEN: I'm guessing they fixed the clock speed problem. ANDREW: I hope so. I actually haven't bought the Steam version because I still have my old versions and I play it in DOS box. JAIM: DOS BOX. BEN: Awesome. Okay, well I've got a couple of picks. My first pick is Apple Care. My four year-old daughter, bless her heart, dropped my iPad. I still love her. One day I’ll let her out of her room – just kidding! She cracked the screen, which teaches me not to give an iPad to a four year-old. So anyway, I went to the Apple store, and it turns out, if you have Apple care, replacing the iPad mini screen only cost $50, so plus one on that. And my other pick is a beer pick, for all of you teenagers out there – just kidding! It’s Rogue Old Crustacean Barlewine. Comes in a super awesome bottle, and one of the kind that has the cork hinge-sort of mechanics on it, so you can reseal it yourself. Anyway, it’s a barelywine, so I don’t know, 18% or something. It’s really, really potent, so don’t drink more than one of these things. And those are my picks – Jonathan, do you have any picks for us? JONATHAN: Yes. Sorry to get all shameless self-promotion, but I did just write a book. Yeah, at the pragprog.com/book/pssprite – of course, it’ll be in the show notes – and I'm quite proud of this. When I am shameless, I do it right, so. Yeah, I think it’s great. We approached game development from a standpoint of “you're not an artist, that’s okay. You don’t have to draw frames to make things work and we’ll practice moving things around and build a few games and we’ll see how it goes,” so it’s good stuff. There's a site called www.sprite-kit.com – that dash is not, I'm just sounding out the dash. And I think it’s en-dash, not an em-dash – I'm not quite sure how you do that. But yeah, that site is a fascinating place to grit a lot of up to the minute Sprite Kit stuff. That’s where I found out about a lot of tilemap applications that work with Cocos and how they're now declaring that they're supporting Sprite Kit and stuff like that. It’s a great resource to find stuff out. I also just mentioned that Apple’s documentation for Sprite Kit is unreal. They have a full adventure game that the wrote as an example and it’s very complex and hard to get into unless you can know your way around, even though they have – they’ve actually documented how they wrote it; it’s a fascinating experience. It’s one of the best set of documentation pieces that I've seen come out of Apple for a brand new technology that they just dropped on us; it blows my mind, so we’ll put the link to the Sprite Kit dev guy and the Adventure Game Example walkthrough – it’s really impressive. And of course, the WWDC sessions on Sprite Kit – you do not pass go, do not collect $200 – you must check those out. And, and, I will add one more, since Andrew –. BEN: Bonus pick! JONATHAN: Bonus picks! Piggyback on what Andrew said, Commander Keen – ah, so many good memories. And the Goodbye Galaxy – Commander Keen Goodbye Galaxy was my favorite out of the series, so I’ll give that a series thumbs up. ANDREW: Yeah, I think four and five were the best. I was a kid when I played these, and they used to have these hint lines where you'd call a number and pay like $1/minute and some guy on the other end would tell you how to get through a part of the game that you're stuck on. And it’s just funny to me, because now, of course, there are a million walkthrough sites online, but I used to go out and tell my mom, “I'm stuck, I'm stuck!” And I’d call the hint line and I'm going to be on the phone for five minutes and it’d cost $5. Sometimes, she would let me do it. BEN: Yeah, Goodbye Galaxy was the one I remember most fondly. Those were some good games, man. JONATHAN: It was an era. [Chuckles] JAIM: I was in the library getting the book that had the cheats if I needed to. JONATHAN: I just cracked open a hex editor and had at it. JAIM: Hardcore, hardcore. JONATHAN: Good times, good times. Like Oh, there's the money value. Alright! BEN: Okay, well if that’s the end of the picks, thanks a lot for coming, Jonathan. I really, really appreciate it. Sprite Kit’s a fascinating topic and I look forward to checking out the book. When does it come out, by the way? JONATHAN: It’s in beta right now and we’re probably going to go – I'm finishing up the last chapter right now; we’re probably going to go to print in the next couple of weeks, and then it’ll be out on the shelves. I guess, I don’t even know if there are bookshelves anymore. But it’ll be out soon. BEN: Cool. Alright, thanks!