037 JSJ Promises with Domenic Denicola and Kris Kowal



02:41 – Promises

  • Asynchonous programming

05:09 – Using Promises from top to bottom 07:08 – Domains

07:55 – Q 10:22 – q.nfbind 11:15 – Q vs jQuery

15:41 – long-stack-traces

19:36 – Original Promises/A spec and Promises/A+ spec

24:22 – .then

26:58 – Nesting Promises

  • spread method

28:38 – Error Handling

32:57 – Benefits of Promises

  • Error Handling
  • Multiple Async at once
  • Handle things before and after they happen

40:29 – task.js 41:33 – Language

44:11 – Mocking Promises 45:44 – Testing Promises


This episode is sponsored by

comments powered by Disqus


JOE: I can’t imagine your baby face with a beard, Jamison. JAMISON: I never thought I had a baby face. AJ: It was always a man face to me. JOE: Everybody who is 15 years younger than me has a baby face. [This episode is sponsored by ComponentOne, makers of Wijmo. If you need stunning UI elements or awesome graphs and charts, then go to wijmo.com and check them out.] [This show is sponsored by Gaslight Software. They are putting on Mastering Backbone training in San Francisco at the Mission Bay Conference Center, December 3rd through 5th. They’ll be covering Jasmine, Backbone and CoffeeScript. For more information or to register, go to training.gaslightsoftware.com] [Hosting and bandwidth provided by the Blue Box Group. Check them out at bluebox.net] CHUCK: Hey everybody. Welcome to episode 37 of the JavaScript Jabber show. This week on our panel, we have AJ O’Neal. AJ: Yo, yo, yo, comin’ at you live from the executive boardroom suite of Orem, Utah. CHUCK: Jamison Dance. JAMISON: Hey guys! CHUCK: Joe Eames. JOE: Hey there! CHUCK: Merrick Christensen MERRICK: What’s up. CHUCK: I’m Charles Max Wood from devchat.tv and this week we have some guests — and that is Kris Kowal. KRIS: Hello. Yeah, Kowal. CHUCK: Kowal. OK. And Domenic Denicola. Did I say that right? DOMENIC: Denicola. CHUCK: Denicola. DOMENIC: It’s OK I got Americanized. That’s probably the proper Italian pronunciation. Hi guys! CHUCK: I speak proper Italian, so probably. KRIS: Yeah and for what it’s worth, I think that the proper Polish is Kowal or something, but yeah. JAMISON: Kris, are you from the Midwest? You have kind of Minnesota-ish accent. KRIS: No. I’m actually unfortunately from somewhere in the suburbs of Los Angeles, but I grew up indoors and did listen to Prairie Home Companion. So I don’t know. Maybe. [laughter] CHUCK: Awesome. All right. So this week we are going to be talking about… actually there’s one thing I need to announce before. If you are listening to this episode, you’ll probably notice a little bit of a difference with our sponsorship message. I actually left off one important piece to one of the sponsorship messages and that is for the Gaslight software training that’s going to be in San Francisco, if you wanna sign up, go to training.gaslightsoftware.com and you can sign up there. They’ve been a terrific sponsor and I feel kind of bad that I botched that. But anyway, make sure that you go sign up if you wanna get their Mastering Backbone training. All right well let’s get in to the show. So we are talking about Promises. And myself being a slacker, I didn’t actually go look up and see what Promises are and how they are of used. So, I’m wondering if one of you guys can explain what they are. KRIS: Yeah I think I’ll take that. A promise is sort of abstraction for asynchronous programming. It’s an object that proxies for the return value or the exception thrown by a function that has to do some of the asynchronous processing. So, it fulfils some of the same of role as callback, and it also does some… they are handy things in best practices for asynchronous programming underneath the hood like guaranteeing that callbacks are actually asynchronous — they are not going to happen in the same turn. And it also does implicit error propagation just like an exception that’s populated back to the stack until it finds a place where it can be handled. DOMENIC: Yeah. It’s a very powerful abstraction mainly because it takes all the synchronous concepts and latch them back. So with a callback, you never know what’s going to happen exactly because it’s so raw and low level, it’s completely up to the control of the library. But with the Promise, kind of rebuilt on this guarantees that the callback or you only see the result once — you only see either the results or an error. And if there’s an error, it will propagate (as Kris said), which is awesome. That type of thing. CHUCK: So propagate how? I don’t completely follow that piece. KRIS: Yeah. In the same section — in the same census of thrown exception, if you throw an exception from a function call and the parent function doesn’t have a catcher, it will continue up the stack until it gets to a try-catch block where it can be handled. Similarly with Promises, you can chain Promises and if you don’t put an error handler on one of these chained calls, the error will pass through that until it gets to the error handler. CHUCK: OK. DOMENIC: So more completely in code like, when you are writing callback code, you get an error, and you are like, “Oh, well I’d better check that and pass it up manually,” or maybe you forget or maybe you are just like banging out code and you don’t touch the error, or maybe you are trying to destroy it because you are sick of passing errors of the stack, but you always have to handle it — and it’s such a pain. In Promises, if you write your code that way, if all your functions return promises instead of calling callbacks, then they talk to each other in such a way that you can get this propagation through your functions and not have to deal with each step of the way dealing with the error. JAMISON: So, that right there is like the gist of my promise using Promises in real world projects. This is only in the brief cursory exploration, but I’ve been writing lots of code without Promises and it seems like it’s kind of an all or nothing thing. Like you either have to retro fit large portions of your code base to use Promises or they don’t value very much. Is that your experience or am I just doing it wrong? DOMENIC: I totally agree. KRIS: Yeah. It is an infectious idea — that you get the most bang for your buck if you are consistently using Promises throughout your system. However with Q, in the last year I think (I don’t remember when we did this) but started adding interfaces to make it easy to bridge between callbacks and promises, so that you can really easily call functions that use the Node-style continuation passing or back and forth. So you can go from Node to Promises and Promises to Node as cheaply as possible. But it’s true, yes — you get the most from Promises if you are using Promises from top to bottom. Especially since like there’s this layer with Promises where you’re interacting with callbacks inevitably and it’s usually very low-level. And at that layer, using Promises is actually kind of inconvenient. Domenic, can you take over? DOMENIC: Yeah sure. I mean yeah, it’s nicer to have them all throughout your system. And the benefits there are kind of enormous. (We’ll talk about that more.) But you know, you do need to write little wrappers around whatever your traditional async code or whichever at the level you wanna introduce you promises. So you can certainly use Promises for all your application code — it’s just then you need to use this little q utilities or similar at the boundaries where you’re interfacing other people’s code. Or you can just buy in to the whole promise ecosystem and go look for some promise using packages and then use them. In terms of the benefits, I do wanna mention this. So I went to NodeConf SummerCamp recently and we had these whole discussions about domains, which apparently have been worked on in Node for a year send this method of dealing with exceptions, so that if you throw an exception in your program, you can kind of tell where it came from. And in particular, what HTTP request it belongs to so the user can get a 500 instead of crashing your server. And what I was amazed at was I realize it was completely unnecessary in our HTTP server because we are using Promises. So, we got the normal exception bubbling behavior all the way back to the top of our request handler where like, “Oh, somebody threw an error somewhere down there. I don’t need any of these domains. I just got this error and I’m just going to show it to the user in 500 page.” KRIS: Yeah. CHUCK: Huh. That’s interesting. So you keep talking about “Q” — what is Q? KRIS: Q is a project that I took over vaguely about two years ago. So it’s a promise API. It’s an NPM you can just NMP install Q. It also works as a script so you can use it in the browser and introduces a Q global variable if you do it that way. And it just provides ways to make promises and then once you have them, you can start playing with them through the interface. The original version was written by Tyler Close at Hewlett-Packard. He’s object-capability security programmer — now working at Google. And it comes from this distributed computing world. Distributed computing was object-capability programming, so it’s like this information hiding type of programming. And so out of that same group of people the Google’s Caja project came out — that’s the sandboxing of JavaScript, so that you can have virtually suspicious programs running in the same context, like log-in credentials and widgets. And also, a lot of the stuff that came out ES5 came out form that group of people. From my understanding, it comes mostly from the work with Mark Miller who’s now at TC39. It has a really great academic heritage and so I took it up and decided to maintain it for Node and then carried it from there. Domenic started working with me about a year ago. DOMENIC: Yeah. From a more practical perspective, as somebody who just joined the project about a year ago and doesn’t have all the historical and academic heritage, Q is just like the most awesome promise library. So promise aren’t like something built in to the language but they’re very simple thing you can implement if JavaScript. And Q is just the most full-featured and the most like thought out and in some ways the most complicated and the most bloated but I think at 3 Kilobytes, it’s not that bad. It’s got all the things I want. And now that Kris is giving me codexes I just add more things when I want them. So the point is that if you wanna use Promises, Q is a great place to start because it has all the toys. Later you are like, “Oh I only use the subset of Promises.” Maybe go check out a smaller implementation but I think Q is great. JAMISON: So Kris talked about the abstractions you added to make it easier to work with Promises from callback APIs, can you talk a little bit more about those? KRIS: So they are pretty simple. My favorite is “q.nfbind”, which is just a funny way of saying like take this Node callback and or take this Node function which uses callbacks and turn it in to a function that returns promises. So you just pass it like fs.readFile and you get a version of fs.readFile that returns a Promise. And so it has the same behavior like if fs.readFile is going to call you back with an error, it will return a rejected promise — is what we call it — a promise in the error state. But then you can just easily adapt it by this one line little functions to create it. JOE: Very cool. So what’s like the difference? I think that most people who do use Promises probably use jQuery’s promises because that is the most ubiquitous language that actually implements Promises. What would be some of the main differences between what Q does and what jQuery does? KRIS: I can take that. The biggest difference between jQuery’s Promises and Q’s Promises — and for that matter a lot of the other — I guess we’ll mention the commonJS Promises later. But the biggest difference is that a jQuery Promise can receive multiple Promises as an argument and then spreads them out into the arguments of the handler. So, instead of modelling a function call, it models array of function calls and allows you to handle all of them at once. We provide the same facility in Q in a slightly different way, with the function that takes an array of Promises and provides an array of results. The other really big difference is in chaining. And the nice thing about having this abstraction that only models a single function calls return value or thrown exception is that when you chain on a Q Promise, you get a new promise for some further work and its single resolution. With jQuery’s Promises, the “.then” method with is sort of our main method for working with Promises, you don’t get that clean abstraction. Each of .then is just chained in a jQuery way where it returns this. With the Q promise, it returns a new promise for whatever is returned by the callback or errback. In all fairness, jQuery does provide another method that lets you do that, but it’s not as convenient. DOMENIC: So actually, that’s a little bit out of date. JQuery fix themselves in 1.8. Their then method is now the same as their pipe method and it does do chaining. But they have a fail flaw in their chaining implementation, which is that they don’t do thrown exception handling at all. So the whole abstraction breaks down when you can no longer throw an exception and have it turn into a rejected promise. They just immediately like say, “I don’t know what to do with this. I’m going to hand it off to the browser.” Or if you are somehow using jQuery Promises and Node like “underscore.deferred” for example, then they will crash your server, so you lose all of the power in my opinion. And so, Kris is more diplomatic about it but I think jQuery promises are unfortunate on the world. I wrote a large essay about that recently – about how some implementations are kind of missing the point of Promises. JOE: Can we get a link to that in the chat note so we can– DOMENIC: Yeah for sure. JAMISON: I think it’s been picked a few times on this podcast. It’s really good. MERRICK: So Domenic, I got a question for you on the rejected with the thrown exception handlers. How does that work? I’ve noticed that when I throw an exception out of a set time out etc. because it’s on a different call stack, you lose access to the error at that point. So even when I’m working with promises, I still have to manually do deferred.reject(). So how does the exception like throwing get turn into rejected promises? How do you guys implement that? DOMENIC: That’s just kind of how you would expect. There is a try-catch wrapping your callbacks, your handlers that you pass to them. And so if you throw an exception there, we catch it and transform it into rejection. And yes, you discover this kind of flaw where if you do set a timeout, it’s like you’re in trouble because you’ve escaped our try-catch. But (this is kind of where the point earlier about you have to bind the promises all the way) if instead of using set timeout, you used a promise based method for delaying your work — like q.delay, which is a utility function — then we’d be able to handle that no problem. MERRICK: OK very cool. I was just wondering because anytime I’m testing out asynchronous control flow, set time out is kind of where I go, so that was unfortunate. DOMENIC: Yeah. That’s a good point. Maybe we should include that in the README. Interesting. CHUCK: So I’m a little curious about how it actually — with the propagation — how that tactually works. And really what I’m driving at is at least in most of the other languages I’ve worked in, (I haven’t done a ton with exception handling in JavaScript itself) when you throw an expectation and it bubbles up, it builds a stack trace as it goes and it sounds like you are doing a try-catch at each level. So how do you do that and not lose the information as you bubble the exception up to the top? KRIS: That’s a fantastic question and sort of leads in to one of the great features of Promises especially with Q’s implementation. Domenic recently added support for long traces, which are asynchronous stack traces. So, what we are doing in Q is keeping track of the stack trace at each event loop turn, where you are deferring to an asynchronous operation. And we can stitch those back together to tell you not only what happened in your current turn of the event, but where you were coming from in previous terms because our Promises chains model a stack trace. CHUCK: So I have one more question. You keep saying “turn”. I’m not sure exactly what you mean by a “turn”. KRIS: Right. The idea is that in JavaScript, you are running an event loop and each time you start a new event that it has its own stack and then the turn ends when you return back to the root of the event and another event gets a chance to run. CHUCK: OK. DOMENIC: So basically like process next in Node will schedule you for the next turn of the event loop. CHUCK: And the next turn is when… so I can handle so many events at once so when I have an open slot then you get your turn? DOMENIC: Yup. JOE: So, one question I have about the way that the chaining works… because  I work in a browser as well where you don’t exactly have kind of APIs into the event loop… sorry the stack trace — the asynchronous stack trace you guys able to build up. KRIS: Yeah. Provided that you are on an engine that provides stack traces that are manipulable. Yeah so if you are running in Chrome with V8, we can basically we are able to do your long stack traces for you on the browser as well. JOE: So what about IE and Firefox, Mozilla? KRIS: Domenic would have to tell me if he did anything about Firefox. My suspicion is no. It does provide a stack trace but it’s not manipulable — it just gives you a string. And I don’t know anything about the latest stuff from IE. DOMENIC: Yeah. So we previously were using V8 manipulate stack trace APIs – the Error.captureStackTrace. It’s really fascinating set of APIs if you wanna check it out — Error.prepareStackTrace. But we moved away from that in just the latest release to directly manipulating the strings, because it turns out those APIs leak memory and it was just a bad thing all around. It was like, if you touch the stack property too early or if somebody else touches it before you do, then you lose the ability to manipulate the stack trace. It was weird. So we are using strings now. And that does open up the ability of bringing other browsers. I know IE10 has completely adapted Chrome’s stack trace format. So if I just remove some checks that like use capture stack trace, maybe it will work in IE10. Firefox has their own weird format but you know, we could in theory write an adapter for that. So yeah we could make it work everywhere – maybe. CHUCK: Right. So, the yet to be released Internet Explorer version. I just think it’s funny it should support it. But anyway. [overlapping talks] MERRICK: So Domenic, do you think you could elaborate a little bit on the Promises/A and the Promises/A+ specification you kind of been working on? DOMENIC: Yeah for sure. There is a fun story there. So as I mentioned, I wrote this essay about You’re Missing the Point of Promises. And what happened there was Ember, there was this commit to Ember that add like “Ember.Deferred” and it basically was… it was not even a copy of jQuery’s deferred — it was even worse. So that’s why I wrote this big long rant. I made it generic. I didn’t mention too many names. But the idea was I really want to get across that Promises, as captured in the original commonJS Promises/A spec (and Kris can talk about more about that) because he was there and I think wrote it? Or no he— [crosstalk] KRIS: No. Utah’s very own Kris Zyp on did that one. DOMENIC: But the point was that the commonJS Promises/A spec have a lot of wisdom in it. And that wisdom was condensed down into like 2 or 3 paragraphs that everybody keeps overlooking. And so if they just paid attention, things would have gone well. So what came out of this is kind of like, this has been brewing for a while because we have this defective jQuery implementation in the community for years. And so, what came out of this was collaboration between a number of promise implementers, myself and Kris from the Q side, but it was really lead by Brian Cavalier (I’m not sure how to say his last name) but he wrote the Promise library called “When.JS”. And also we joined forces with Yehuda Katz who had recently written RSVP.js. And we said, “OK let’s get this settled once and for all. Let’s write a spec that takes the original commonJS Promises/A spec, makes it more fleshed out, kind of standardize it on some things we’ve learned over the years, some obviously missing features,” and puts them all together. So that’s actually like, I wouldn’t say it’s finished, but at this point we are just tweaking the wording. So it’s pretty good and I’m very happy with that. And the next step is to finish writing a nice test suite so that everybody can test like an executable test suite, so that you can say, “Oh, I’m compliant with Promises/ A+.” Well I already wrote one for Promises/A. JAMISON: I was going to say, you wrote a test suite already, right? DOMENIC: Yeah. Actually that was the part of the story I forgot and skipped over. So after writing this I’m like, “Well how can I actually not just make this a rant on the Internet? We’ll let me go off and write executable test suite for Promises/A behavior and for some common extensions.” So I wrote that and I actually got a lot of trashing. People really enjoyed making the libraries pass that. And one library I mentioned earlier, the underscore.deferred — which is a part of jQuery’s — they recently like went the trouble to conform to the test suite. So now we have something that’s started with this kind of crazy jQuery approach and it actually went and conformed to Promises/A, so I feel like I actually did some good in the world. JAMISON: That phrase like, “how can I make this not just a rant on the Internet”, that was amazing. That would like change the world if everyone did that. DOMENIC: [laughs] JOE: So what do you think of Underscore’s deferred? DOMENIC: I haven’t looked at it in detail, but I’m just very glad that. Wookiehangover is the guy behind it. He was totally willing to make it work in kind of standard — way. So it’s awesome to bring something… I guess for people who are coming from jQuery, it’s a nice bridge between worlds. It’s got all the same kind of familiar APIs but it’s actually standard compliant. JOE: So I’m not familiar with that Underscore.deferred. It’s not part of the regular Underscore library, is it? DOMENIC: No it’s a plugin for Underscore. And the idea is that it’s just a standalone version of jQuery deferred that you can use in Node or browser without using all of jQuery. CHUCK: So, can I ask really quickly — if you write a test for this spec, how do you do that without specifying what the API should be? Or do you specify what the API should be? KRIS: There are two levels of the API. One of them is that the promise library needs to give you some primitives to construct Promises of various kinds, like rejected Promises and fulfilled Promises and deferred Promises. And the implementations don’t have a consistent API for that, so Domenic uses adapters for each of the implementations on that level. But once you got a promise, the promise has a standard API and that’s what his tests would test. CHUCK: OK. DOMENIC: Yeah. And you can add on extensions to Promise API convenience functions or whatever, but the core functionality of this then method needs to be consistent. And the reason actually for that is not just because we like standards and think everybody should be the same, but actually because that way you can interoperate your promises. You can say, “Oh, well I’m in a callback from a Q promise in a handler but I’m going to return a when promise and that should work.” My favorite example of this (it’s a little esoteric), I wrote an assertion library that lets you make assertions in your tests about promises and it just uses the then method and doesn’t depend on any promise implementation at all. So you give it any promise that is complaint and has a good then method, it can do crazy stuff to chain off of that and create a  series of asynchronous tests for whether that promise is in the right state. So that library is called “Chai as Promised”. And I’ll put a link in that show notes. CHUCK: I like it. I think it’s cool. And looking into Promises and the then… I don’t know if I wanna call it a method or a function, but it just it seems kind of nice as opposed to nesting your callbacks so that its, “We’ll call this back and then when you are done with that then do this other callback and then when you are done with that…” because you get this weird nesting and it’s hard to think about and .then seems to just clear that up so it’s— KRIS: Right. The then method gives you the opportunity… it gives you a choice – you can either nest or you can chain. And it both works in most situations are the same. There are situations where it’s compelling to use nesting because you want to capture multiple values in your closure and then combine them to provide a new result. So it doesn’t completely replace nesting callback hell, but it does give you an out if you want to flatten it. CHUCK: Right. So is it solely… I don’t wanna say “one dimensional”… two dimensional where you can only say, “Do this callback and this callback and this callback.” Or can you do it so that it’s you know, like you said you are going to go pull multiple sets of values and combine them or you want to log to several places at the same time. KRIS: Yeah you can have branches in your control flow as promises. DOMENIC: The thing to keep in mind is it’s like synchronous code — that is what it’s trying to model. So, you can have branches in your synchronous code where you take a result and you do multiple things with it. Most of the code you write probably doesn’t do that, right? It probably just use in the next stage of the calculation. But eventually maybe you wanna log in and display it on the screen — who knows. So you can definitely do that. You know, most of the code you write, it doesn’t need to catch exception, you catch those at the boundaries so you don’t pass them up all the time. That’s kind of the compelling part there. Most of the synchronous code you write, you don’t need to capture things from multiple calculations and combine them, so you don’t need to nest your promises there. But if you do need to like do this and this and this and then put them altogether, then you need to kind of nest your promises to get them on the closure. JOE: Yeah and about Q, I mean obviously you guys know more about it than anyone, but I’ve noticed they have a cool like spreads and joints and things like that that make working with multiple Promises… you can kind of compose multiple Promises in to these unified Promises, which are really cool. KRIS: Right. The spread method is our way of doing the nested Promises without actually having to nest where you can take an array of Promises and then receive them all as arguments. Yeah so you can flatten things up pretty easily. DOMENIC: That is actually like the big difference between – well, its two big differences — between synchronous code and asynchronous with Promises is you can do two asynchronous things at once. So you need a method like q.all to aggregate the results and say, “OK, wait for both of these to be done.” And then other big difference is actually like the end of the chain of error handling but I don’t know if you wanna dive into that. It’s a bit technical. CHUCK: Go for it. DOMENIC: All right. This is the dark side of Promises – it’s the dark side of exceptions too, but in Promises like they make the… or it’s the dark side of callbacks rather, too. But in Promises, we make the error handling story so easy that you kind of forget you have to do anything, but the trick is, so you are bubbling up your exception. And in synchronous code, it’s get the talk and you are like, “Oh. Well, nobody caught it. Let me just crash the program or put it in the window.onerror,” or whatever. Great, you can handle that. In asynchronous code, somebody might come along to say, “Oh, I’m going to check out that Promises’ state and handle it later.” So if you bubbled up all the way to the top and nobody is listening, you can’t guarantee that you should like throw immediately or crash the program or go to the console, you can’t say– [crosstalk] JAMISON: Because someone might pass that promise into some other handler or something. Is that what you are talking about? DOMENIC: Exactly. Exactly. And that’s the idea is promises are first class values, so you can just pass them around through your functions. So if you are not careful, you can swallow errors this way — and this is the sad part. And I don’t think it’s that big of a deal like I said because you can forget to do if error in your Node callbacks but if you– [overlapping talks] We have a solution for this and Kris can talk about it, but that’s the problem in a nutshell. KRIS: Yeah and we have a partial solution. I’m not about to claim that everybody is happy with it but [chuckles] but there’s a dark side for sure. The idea is that for one, we have a best practice. If you are providing Promises as an API, you just return the promise. But if you don’t return a promise, you have to terminate it with a call to .done. .Done receives a promise and then if that promise is rejected, it propagates an exception so that it behaves just like it had been bubbled all the way up to the top of the event and been thrown in the global context. The problem with that is yes we can end up with these free floating rejected promises lying around. We have a partial solution for that for some browsers. And that is that we are taking advantage of the fact that if you log an array to the console in certain browsers, you’ll get a live representation of that array. So what we do inside of Q is that every time you have a constructor rejected promise or promise becomes rejected, it gets added to this array, so that you can inspect the error stacks for any unhandled errors. But if somebody asynchronously handles that error, we remove it from the array. So you get a view of all of the presently unhandled errors as your program is running. We have some ideas for making a more general solution for that. Domenic has called out to make a browser extension. And we need to do this anyway because one of the powers of Promises is that you can create this object graph that represents all of the outstanding work that your program is doing as a graph and you can just say, “This Node is waiting for this Node and this Node is waiting for that Node,” and you look at it and you get an error stack trace for any of those nodes. There is a project from an older implementation of Promises called “causeway”, which creates a visualization like that and it also has a multi-language protocol so that it can visualize these… it’s a  great debugger for asynchronous code that gives you all of the stack traces of all pending work, basically. JOE: It’s called what? KRIS: Causeway. JOE: Cool. KRIS: Playing on “causality” because all of these promises occur in these causal chains of this because of that, because of that, because of that. JOE: Sure. KRIS: So we need to port that into the Q ecosystem at some point. And that’s one of our long term goals. JOE: Very cool. JAMISON: So, I just wanna take a step back and let you hopefully answer an easy question. So it seems like lots of the benefit in Promises over callbacks is in un-nesting that pyramid of doom, where you have lots of async operations in a row and people talk about callback hell in Node or in the browser. And I always think that’s a bit of a straw man argument because that’s what you do by default if you don’t know any better. But most of the time, you don’t need to nest asynchronous functionality that much. You can use named functions or you can use like control flow library. So I haven’t found that much pain in callback hell in a long time. So, can you just talk about again like what the big benefits over just using callbacks and managing callbacks on your code are? KRIS: I’m going to put this right out is that, solving the pyramid of doom is not benefit of Promises. It’s the most appealing superficially because it makes it easy to go from using anonymous functions in one style to using anonymous function in a flat style. It is a superficial argument. The real benefits of Promises are… well for one, there is the error propagation and implicit propagation. I think that’s probably the biggest advantage you get. DOMENIC: Yes! Agree. KRIS: And after that, there’s this distant dream. I’m not going to pretend that promises as what we have right now are fulfilling the dream, but it’s really about better ways to debug distributed programs. And part of that is that using this promise, you get an implicit object graph which represents domains, it also can represent control flow for multi-process programs. We don’t really have a multi process debugger yet and Promises are the primitive we would need to be using in order to take advantage of a [inaudible]. So the way that works is in the long run. I’ve got another project called “Q-connection” (formerly called Q-con), which lets you use Promises as proxies for remote objects. And this is really where the real strength of Promises shines. This is the thing that you can’t do with callbacks. So supposed that you request an object from another process, right? You would immediately [inaudible] for that remote object. What if you could immediately start sending messages to that remote object even though you don’t have a physical copy of it yet? And even you don’t even know whether it’s going to be fulfilled or rejected in the future, but you don’t wanna waste round trip times. You want to start sending messages to this object now so that they can be processed on the server as quickly as possible — and so that you can get response as quickly as possible. That’s what Promises are designed for at the core. They are a proxy for a remote object to which you can send messages. The most useful thing about them is that you can call that .then on them and get resolutions in the same process. But the real big benefit is when you’re requesting a remote object and you want to communicate with immediately through pipelining of these messages. So in addition to the .then method, Q Promises have a .get method, a .put or set method, post and these all correspond to primitive manipulations of objects, like getting a property of an object or setting a property of an object or calling a method of an object or like the invoke method. This level of abstraction says that if you are working with promises as proxies for other object, you can pipeline messages and get responses as quickly as you can without causing the problem with chatty protocols where you’re sending a message, waiting for the response. “OK. Now I got the response, I’m going to send another message.” You know what I mean? JAMISON: So that sounds way more complex. That almost sounds like… I mean I guess it’s funny because its named Q, but it sounds like a message passing like queue-based service where  instead of doing that with your architecture, you are doing that with individual objects. KRIS: Uh-huh. JAMISON: That’s nuts. That’s super cool. CHUCK: Its nuts but it’s extremely powerful because you can effectively manage and customize the callbacks on an individual object basis. Where with most queuing systems, you just set up some kind of helper or worker that does a very specific job and does it well, does it over and over again. JAMISON: I’m falling through that tunnel in 2001 with Space Odyssey right now. [overlapping talks] DOMENIC: Let me pull you back a little bit, because Kris jumps straight from the like you know nice little, “Yeah, you get error handler,” to “Whoa! We have this amazing dream.” And that is very cool, but I haven’t gotten a chance to play with that yet, so I’ve just been stuck getting all the benefits back on earth. [laughter] CHUCK: So here in the real world… DOMENIC: So for me, the error handling benefit is huge. And that kind of comes with the theoretical benefit, in that it parallels your synchronous code so it’s very easy to think about that, right? But with the callback, you never know. Like is it going to give me two arguments or three or who knows? It’s not going to be like a function that just returns a value. And you know, you get this classic bugs and this gets discovered weekly where somebody called their callback twice or called their callback with an error, but then also with a value or that they threw an exception so their callbacks never gets called, right? You get these kind of guarantees with Promises that it acts like a function call. It will always come back with single value or a thrown exception and never both, and never twice. So that’s a big benefit — just kind of theoretical though. The other thing is that when you are doing asynchronous code, you are going to need some basic joining primitive, something like async.forEach or async.map or whatever. Q has that in a very nice and kind of unified form. It’s not just these ad hoc methods that take arrays of arrays of functions and arguments to patch those functions and so on. And I think that the Promise API, where you get first class object representing the result of your computation is what allows that to be true. And finally, to go in to the atmosphere perhaps, if not the stratosphere, because Promises give you the synchronous to async parallel, they provide the foundation for the future. In ECMAScript 6, we are going to have generators, which are an ability to exit your function for a while then comeback in. So this is mostly used the examples are like sequences where you generate a sequence and you yield and then you come back in when somebody calls you the next time. But because they allow you to exit function and come back in to them, it’s exactly what you need for asynchronous code. So there’s a really great concept of using generators plus promises to give you this kind of synchronous looking, but still very clearly delineated as async code using this yield or in ECMAScript 6. So we’ve actually already do this in Firefox, but it’s not working at any other browser yet and who knows when Chrome is going to get around to implementing generators or whatever. But this library is called taskjs.org by one of the people in the Mozilla team. And that’s my dream is to get that working everywhere. KRIS: Yeah. So we are talking a little bit about driving the future of JavaScript. Q is coming from a heritage from Mark Miller who is on TC39 and proposed a concurrence for the next version of JavaScript, which introduces additional  syntax to make working with Promises simpler. And Dave Herman took up that and put together TaskJS, which composes Promises with generators to create sequential looking control flow using Promises and everything. So you can yield within a try and you can catch an asynchronous exception with a synchronous try-catch block. It is a very, very beautiful abstraction. AJ: So I don’t know too much about the generators, but I’ve been a long time believer that the best place to solve the Chrome’s problem is definitely in the language itself. And I wish that, as they have been working on CoffeeScript, I wish that they would have put something, be it like Q or like what jQuery has or async or futurejs whatever, I wish they have put something like that in CoffeScript because it really belongs in the language. KRIS: Right. If you take a look at a language that has Promises as a primitive, there is “E” which is the language that Mark Miller put together before he started working with JavaScript. And in E, Promises are primitive and you can send messages to them directly using object-like notation. And that’s what Mark Miller is pushing for in the concurrency straw men — more primitives in the language. E also has a primitive when catch blocks to resolve promises just like our .then methods except with language-level syntax. We are entering an age of transpilers, so I imagine it would be pretty simple to embed a promise abstraction inside CoffeeScript. I know why CoffeeScript didn’t do it. CoffeeScript wanted to be as close to JavaScript as it could be without a runtime… to avoid [inaudible] now. This one had a huge runtime that came with it. Part of CoffeeScript success is that it doesn’t come with a runtime. AJ: Well what I’m saying is like with CoffeeScript, you could take a library and you could just make it so that every single function returns a promise object. You know? So that it’s just standard. Because the reason that I myself don’t use Promises very often, like I use it for async and I use it to join multiple callbacks because that actually simplifies my workflow by going back to original statement, in order for Promises to really be effective, you have to be extremely vigilant about it. You have to do it the whole way through your framework, from the ground up and there’s just a lot of boiler plate you have to introduce in every function you have to create the Promise. You have to return it, you have to do something with it whether its synchronous or not and if you really want it to be composable, then everything has to be a promise even if its synchronous so that you really get the full benefit of it all. KRIS: On some levels I agree, on others I don’t. I don’t think that every functions needs to be asynchronous and a lot of functions don’t benefit from it. But, on the other hand if it were something that the language optimized, yes it could be better. For sure. JOE: So I do have a question about. I was trying to do some testing with Promises and I was actually mocking an external dependency that had a promise interface to it. And so, I was using Sinon.JS and trying to replicate an object that actually had a promise in Sinon and it ended up being herculean pain in the butt. I don’t know if you guys have had any experience in mocking promises and how many advice for that, but if so, that would be great. DOMENIC: Yeah. So I did this all the time. Yeah Promises… I guess maybe we should write this up somewhere. Like got into testing with Promises but, there’s a few levels; but it sounds like especially if you are just trying to mock an API, all you need to do is include a Promise implementation like Q and just return a promise that is already fulfilled or rejected with whatever you want. So I have tons of things in my tests that just say, “OK take this API and stub it out to always return fulfilled and then test the behavior is what I want. And make it always return rejected. And then make sure it always does what I want.” And so, it’s not that bad. I mean, it sounds like you are maybe I mean re-implement a completely fake promise with no helper, which would not be fun because then you have to follow the spec and then you start implementing a promise library and that takes like at least 60 lines to codes. So, yeah. JOE: Well it wasn’t quite that bad, but you’re idea is brilliantly simple and I feel stupid for not having thought of it. DOMENIC: Yeah. The other things about testing Promises (and I spend a lot of time on this making it easier at least for myself, and some other people seem to use it too) is that it’s really natural to return your promises as the last thing you do in any promise function — but even in your tests. So your tests will succeed if the return promise is fulfilled and will fail if it’s rejected. And that’s not supported by very many test runners. The only one I know is natively supported by is “Buster”, but I don’t know anybody who uses Buster — maybe that’s bad. I don’t know. It’s probably a great testing framework. The one I use the most is Mocha. [crosstalk] JAMISON: I like other ones more than Buster. It’s probably the nice way to say it. [laughter] KRIS: Oh that’s fine because we have solutions for them too. [laughs] The Jasmine in particular has a pretty pluggable architecture. I managed to make it so that it blocks would accept promises as returned values and like maybe 10 or 15 lines. So, I know that Domenic has done some other work for other assertion libraries and take it away. DOMENIC: Yeah. So my thing is “Mocha as Promised”. So Mocha is my favorite test runner these days. I try some of the pool requests like no I don’t like promises, I’m a callback guy. And you know, after — it up plus ones and realizing it was never going to happen, I just went off and I duck-punched the hell out of Mocha, so now it returns my… so it does not have a very pluggable architecture like Jasmine. I actually do some amazing tricks where… because they don’t encapsulate any of their variables, I create a setter on the prototype of the test suite, it intercepts when Mocha sets the function and then it wraps it with another function that works better with Promises. It’s awesome. And some of my favorite code I have like a 40-line comment block (maybe not that long) like 10-line comment block explaining how this works. But it works and its fun. So the upside is yeah, you can do exactly that. The other thing I did is “Try as Promised”, which I mentioned earlier. The idea there is that try is an assertion library that lets you say kind of very fluid assertions like you know, “This should be OK. This should have been called with 30, whatever.” I just added a little plugin to it so you could say, “This should eventually be OK. This should eventually div=5,10,20” and that gives you a promise which  you can then use with Mocha’s promise that just return it. So your assertions about your promises end up looking very nice and almost synchronous. Again like some people say you shouldn’t (and I agree) that you shouldn’t make your async code look too much like sync code, and that’s true but if you just introduce like one little signifier like eventually, or a wait or a yield or whatever, that I think is the perfect medium. JOE: That’s awesome. CHUCK: Yeah. I really like it — should.eventually.equal. Well I think we’re running out of time here. Are there any other critical parts of Promises or Q that you wanna go over that we haven’t hit on yet? Or anything that is coming up in the future that we should talk about? KRIS: No. I think we’ve laid out where we are and where we are going pretty effectively. The one thing I can do is encourage you guys to all go check out the Q README. And James Burke, one of my contributors on Q recommended a long time ago that we needed a graduated tutorial because it’s a sufficiently complicated concept, so I replaced the entire README with step-by-step introduction to Promises and I think that you guys will all enjoy that. CHUCK: James Burke… I know I talked to him before. KRIS: RequireJS. CHUCK: Oh. Yes. That’s where I’ve talked to him before – on the show. [laughs] All right cool. Well, then we’ll go ahead and wrap this up. We’ll get to the picks. AJ, what are your picks? Jamison, what are your picks? JAMISON: I only have one. It’s called Code Triage — codetriage.com it’s a cool service that somebody made to basically subscribe you to triage some issues of different GitHub projects. I think it works by finding the projects that are like most watched, with the most issues, has some heuristics like that. And it basically has lots of popular projects and if you want to it’s a good way to kind of make yourself get involved in these open source projects, if you are not really sure where to start. You log in with your GitHub username, password and stuff (well, ID. You don’t give them your password) and then you subscribe to projects and then every day you get a random issue – random unclosed issue — from this project emailed to you. You can check it out, close it if it’s old or help work on it if it’s new. So that’s been kind of fun. I’ve been doing that for a week and it’s been cool. I’ve learned stuff about all these projects even if I haven’t contributed tons to them yet. So it’s just codetriage.com CHUCK: Awesome. Joe, what are your picks? JOE: So I’ve got three of them. The first one is The Creative Sandbox Guidebook. And I’ll be honest I actually have no idea what it is. I haven’t looked at in very closely to what it is. It think it’s something about promoting your business with videos or something but it’s this website built by Google and it’s the coolest thing because literally like a flip open book, you know that’s like pop out books that you find for real. It’s like one of these implemented on a web page and its way cooler. So it’s just visually just tickles my fancy. So, I wanna pick that. I’m also going to pick Steam because they’ve just finished up their Thanksgiving Day sale and at the end of the year they will have their Year-end sale they always have. And that’s awesome because I’m a compulsive game buyer, so I picked up like 4 games on the average price of I think $7. They were really awesome games. JAMISON: I escaped unscathed from the Steam Thanksgiving Day Sale. CHUCK: [laughs] JOE: I did not. I have my copy of Tropico 4 I’ve been wanting for quite a while and several others. And then the last pick I’m going to pick is pluralsight.com, which is not a wholly selfless pick because I actually do some work for Pluralsight, but I kind of felt like I had missed out some computer science fundamental steps stuff with some data structures and algorithm recently, and so I wanted to go find a place to go learn that. I’ve been looking all over these really old 1986 recordings of MIT courses and stuff. And even though I’m an author for Pluralsight, I didn’t realize they actually had a course for it. So I just was amazed at all of the high number of courses that Pluralsight has on all kinds of programming topics once again even though I am familiar with them, I’m still amazed of the things I didn’t know. So my pick pluralsight.com. CHUCK: Nice. Kris what are your picks? KRIS: Check out “montage”. I work on open source framework for web applications, particularly targeting tablet but also great for desktop and it uses Promises and its module loader and it can run and load CommonJS modules in the client side even in development with just — . Yeah check that out and check out the module system for it. It’s montagejs/mr and I think that you guys will enjoy that. CHUCK: Sweet. All right. Domenic, what are your picks? DOMENIC: Gosh. I really wanna tell you about some cool stuff I’m working on for Windows 8 and NodeJS together, with stuff but it’s not quite ready yet. So, keep an eye out for… I’m sure I’ll publicize it. It will be under the name WinningJS. And I guess I have a talk on why I’m doing this because I’m so dissatisfied with the Windows 8 JavaScript Developer Experience. So you can check that out on YouTube. I gave it as CascadiaJS. CHUCK: Nice. All right. So I only have one pick. I bought an iPhone 5 and I am a— [crosstalk] JAMISON: Fan boy. CHUCK: Omnifocus– JAMISON: Oh no, not that. [laughs] CHUCK: And so Omnifocus on iPhone actually has a feature where if you create a to-do or a reminder on your iPhone with Siri, it will actually pick it up and suck it in to Omnifocus and then I get it anywhere. I use Omnifocus which is everywhere. So anyway, I’m just going to pick that feature. There’s a video on the Omnifocus website. Finally we’ll go to AJ for his picks. AJ: So I’m just going to be quick because there’s a lot of people talking here too. But there’s this product called “Buckyballs”. Unfortunately they are quitting their business but they are extremely fun magnet toys. And they have a few hundred left before they close out their inventory. And one of them is its magnet rods with little steel or iron ball bearing things and you can make way cool structures out of them. JAMISON: I think they are discontinuing them because people would eat them and then they like stick together in their intestines. AJ: Yeah that’s it. Yes. But there’s some other companies that are also producing them. But they are just doing the “zen magnets” that are the little balls, whereas Buckyballs actually has several types. CHUCK: All right. Cool. We’ll wrap up the show. Thanks for coming again guys. It was really, awesome. And I’m definitely going to have to go check out Promises. DOMENIC: Thanks a lot. CHUCK: Yeah no problem. Well we will end it. We’ll catch you all next week! [end of podcast]