088 JSJ Lazy.js with Daniel Tao

Download MP3



00:50 - Freelancing Q&A01:21 - Daniel Tao Introduction


Next Week

The Node Security Project with Adam Baldwin


DANIEL:   Alright. So, just the three of us, looks like? AJ:   [Singing] Just the three of us. DANIEL:   [Chuckles] So romantic. AJ:   It is. [Hosting and bandwidth provided by the Blue Box Group. Check them out at BlueBox.net.]  [This episode is sponsored by Component One, makers of Wijmo. If you need stunning UI elements or awesome graphs and charts, then go to Wijmo.com and check them out.]  CHUCK:  Hey everybody and welcome to episode 88 of the JavaScript Jabber Show. This week on our panel, we have AJ O’Neal. AJ:  Yo! Yo! Yo! Coming at you live from the snow. CHUCK:   I'm Charles Max Wood from DevChat.TV. I'm going to do another quick announcement. I'm doing a free or donation-based webinar on freelancing. It’s just a Q & A. So, if you have questions about freelancing, thinking about it, doing it, have questions, you can go to FreelanceQA.EventBrite.com. I'm going to put a link in the show notes so that we could get to it there as well. We also have a special guest today and that is Daniel Tao. DANIEL:   Hey, thanks for having me. CHUCK:   No problem. Did I say your name right or should I have asked first? DANIEL:   Yeah, it’s Tao. [Crosstalk] CHUCK:   Okay. So, do you want to introduce yourself since you're new to the show? DANIEL:   Sure. As you said, my name is Dan Tao. I was just saying I get called Dan Te-yo all the time. So, pronunciation is not a big deal. I am a developer at Google right now. I'm on the Ads Review Team. So, we build a lot of mostly internal tools for ensuring ads quality and things like that. And I do a lot of work on the frontend mostly. I kind of like the full stack but I think by default, because so many other devs don’t like frontend work, I end up working on a lot of frontend stuff. And so, a lot of the time here at Google, that means a lot of JavaScript. And then, in the sort of open source world, I also work on a lot of JavaScript projects and on a lot of Ruby projects. So yeah, I guess the one thing, my biggest contribution thus far is Lazy.js which is a JavaScript library I wrote this year and I'm still working on. CHUCK:   Cool. DANIEL:   So, that’s me. CHUCK:   You're a major contributor to Lazy.js? DANIEL:   Yeah, which is for JavaScript devs. You know, I think most JavaScript devs know about Underscore and you recently had the author of Lo-Dash on the show which is like a fork of Underscore with a lot of optimizations and other stuff added. So, Lazy kind of falls into that same bucket of sort of low-level utility libraries. But the reason it’s called Lazy is because unlike Underscore and Lo-Dash which both eagerly evaluate your function calls when you're using their map produce filter type functionality that they provide for arrays, Lazy does lazy evaluations. So, if you call map filter reduce on an array, it doesn’t create intermediate arrays, it just waits until you try to iterate over the result to combine all of those calls into one sequence that you can iterate over. So, that’s kind of the main distinguishing factor between Lazy and Underscore/Lo-Dash. CHUCK:   So, this is kind of -- the way I think about Lo-Dash and Underscore is it’s kind of the functional set of operations that are convenient to have for collection of things. So, it’s more of a functional approach to things as opposed to object-oriented or procedural or prototypal. So, the thing that I find interesting about this is, evaluating it lazily, is that harder than eagerly evaluating things? DANIEL:   Well, it depends on what you mean, I guess. I would say yeah, it’s kind of harder in the sense of implementing it because whereas with Underscore or Lo-Dash, if you have a method like map, for example, and your input is in an array and your output is in an array, then it’s pretty clear to most developers how to implement. Not necessarily the most efficient way, but at least some way where you create a new array and then you just push a bunch of results onto it and then you return it. Lazy is a little bit trickier to implement than that. But I think at least one of the goals of it is from the user’s standpoint, the developer consuming the library. It’s not really trickier to use because it has almost the same interface. So, if you check out the project homepage on GitHub, you can see the methods are almost one to one sort of like half, more or less parity with Underscore. So, if you're used to Underscore then using Lazy should be pretty straightforward. CHUCK:   I guess the next question is that you said it doesn’t create the intermediate arrays, so to speak. So, does it actually iterate over each one? Let’s say you have a map and then a filter. Does it actually map the first one and then check it against the filter and then do other things? Or does it actually -- what's kind of the procedure for that? DANIEL:   Yes. I mean, it’s basically the fundamental object in Lazy is, I call it a sequence. I actually have a sort of a .NET background. When I first started in software, my first job was in .NET. So, .NET developers are familiar with link. And in .NET, the basic enumerable structure is the IEnumerable interface. So, it’s kind of similar to that in that it’s not necessarily an array, it’s just something that you can get an iterator for. And that iterator is responsible for moving through the collection and sort of giving you each item one by one. So, when you combine a map and a filter, whereas in Underscore or Lo-Dash, both of those methods independently return arrays. With Lazy, it’s sort of an object building approach where when you call map, it creates a sequence whose iterator will walk over the original array and map each element as it looks at it. And then filter creates an object on top of that that its iterator, when it looks at each element one by one, will either pass the element through a callback or not pass it through a callback. And that’s sort of the way that all Lazy has implemented is just different sequence types whose iterators act differently. I don’t know if I explained that very well [chuckles] but hopefully. CHUCK:   Yeah. It just really strikes me as an interesting problem. I think you described it okay. But at the same time, I mean, [chuckles] I have to admit, I'm a little bit distracted too today anyway. But yeah, I really think it’s interesting. So, you basically build the sequence and then iterate through it. DANIEL:   Yeah. So, one of the things that I really like about this approach, and I've mentioned this in my blog and stuff, but it’s that it’s not just about lazy evaluation. It’s actually because Lazy has this concept of sequence which is not necessarily array-based, it kind of opens the door for other things that Lazy can do. So like, I know you also had the guy from IxJS on here recently, the Reactive Extensions for JavaScript. And that’s all about reactive programming. And Lazy can actually do something quite a bit like that. Whereas with Underscore and Lo-Dash, you need an array to start with; with Lazy, this concept of an iterator can apply to things that aren't based on arrays. So, you can apply it to, for example, event sequences. There's a lazy.dom.js file in the Lazy repo that is meant for browser environments. And if you require that file, it gives you a way that you can basically declare a sequence that consists of the events bubbling up from a DOM element. And then, you can map filter reduce over those. CHUCK:   No way! Really? DANIEL:   Yeah, yeah. So, you could wrap an element and call on, like click and then that will give you every -- when you handle a click event in the browser, it’s like an event object that has a bunch of properties like the X and Y coordinates and so forth. So, you could call map on that and then translate that event object to whatever data you want. And then you could call .each. And .each essentially is attaching the handler to the event and then every event that comes through is treated as an element of the sequence. So, that’s something that you can do that’s pretty cool. I also recently introduced something that I think is really neat which is lazy parsing of JSON. So, the typical way you would deal with JSON in a JavaScript app -- and I should preface this by saying this is very experimental, I'm sure there's bugs in this currently but it’s kind of like a proof of concept. But in JavaScript, when you deal with JSON, generally you have to parse a giant string of JSON into an [inaudible] object. And then if you want, you can map filter over that array or that object or whatever. But with Lazy, what you could actually do is, say you have a string that’s a bunch of JSON, you could actually just start mapping and filtering over the JSON without even parsing the whole thing. You just parse one piece at a time. So, that’s another example where you don’t need an array to start with. You can start with other things that sort of lend themselves to the idea of a sequence. AJ:   Why would I want to do Lazy JSON parsing? DANIEL:   That’s a good question. For example, this is maybe a contrived example, but suppose you have a really, really big JSON response that you get from some external API and you know that you just want the first five results. So, if you did it the traditional way, you would parse the entire string which could be, let’s say just for argument’s sake, that it’s a few megabytes long, then you're doing a lot of extra work. Whereas with Lazy, what you could do is just parse the first five elements out of the string and then you don’t even end up parsing the rest because you call parse whatever take five. And then when you iterate over that, it will just walk the string until it’s found five elements out of the JSON and then exit. So, I'll include the link to sort of like a demo what I'm talking about. And another example, perhaps even more contrived but you could also have a JSON string that actually terminates early so it’s actually not even valid JSON. You could still parse the beginning of it. So, that’s another thing you could do. If you go to the demo, you can see there's sort of like an example of invalid JSON and you can see that Lazy can parse all of the elements up until it reaches the syntax error. So, that’s a way you could also iterate and map and reduce and so forth over the beginnings of a JSON sequence without having to parse the whole thing. CHUCK:   So, I want to jump in here because when I think about lazy evaluation, I kind of think of I don’t operate on stuff until I need to pull a result out or until I need to do some work on the entire collection that it basically yields. And so, what you're saying is that part of the lazy evaluation is that I can just tell it how many I care about or however many of these things that I need and then it only evaluates those. And that’s why these other examples work out the way that they do is because I don’t care about the rest, I only care about the first five or ten or whatever. DANIEL:   Yeah. And I think I've had trouble in the past coming up with non-contrived examples of how that’s different. Like I've said, I think on the project’s homepage, it says, “Suppose you want it to map a thousand elements,” and then, “Take the first five.” So, with Lazy, you could call map on an array and then take five and it’s not going to map all thousand elements, it’s just going to map the first five. Whereas with Underscore or Lo-Dash, if you did that, it would create an additional thousand element array consisting of the maps and then if you call take five, it would just walk over the first five. I admit that’s a contrived example because it would just be a bad idea to do it. Like in Lo-Dash or Underscore, you could just take five first and then map over that. But I think a less contrived example would be, if you have some data and you don’t necessarily know what the data looks like, say you're getting it from some external service, say you're getting a list of people and you want to take the first five people whose last name is Smith or whatever. The Lazy way, you could call filter and then give it a function that says lastname=smith and then take five. And Lazy will only bother iterating until it finds five people whose last name is Smith. Whereas with Underscore/Lo-Dash, that approach would filter the entire collection. Potentially say, there's a thousand Smiths in the list, it’s doing a lot more work than necessary. So yeah, I do think [inaudible] a big one of the benefits in my experience to offering lazy evaluation for sure. CHUCK:   So, do you find that you get major performance gains from this? DANIEL:   Yeah. It’s funny. It’s sort of a yes with caveats. So, John-David Dalton who you guys had on the show a while ago, as you know, he wrote Lo-Dash and Lo-Dash is really, really good on performance. A lot of devs who maybe have been using Underscore, they’ll switch to Lo-Dash when they see what a huge improvement they get in terms of performance with that. Actually, the very first issue opened on GitHub for Lazy was John-David Dalton. I don’t even know how he found out about it. I'm guessing he maybe watched the searches every now and then for Underscore clones or something. But he commented sort of letting me know, sort of like [inaudible] nudging me like, “Just so you know, the performance implications of what you're doing here might be kind of worse than you think.” And he sort of linked to some jsPerf examples and showed how lazy evaluation is not always a huge win which is definitely true. And of course, at that time, the library was very young and so, I actually took that as kind of a challenge and my competitive side kicked in and I worked on making the performance a lot better by reducing a lot of the overhead, that lazy -- AJ:   It’s good that it got into your competitive side. [Laughter] DANIEL:   Yeah, definitely. I loved what he said on your show about how competition is a good thing. I actually totally agree. I mean, I totally respect Lo-Dash and I think it’s a great library. And at the same time, I totally want to beat it in performance. So, I guess the short answer is it does give you real performance advantages especially when there's a lot of data. In my tests, I guess pitting the current version of Lazy versus the current version of Underscore, last I checked, Lazy becomes faster when you hit around 20 to 4o elements or more. And when you start chaining together, methods like map reduce whatever, then the results are even more pronounced. But Lo-Dash is still really, really fast especially for small collections like five to 15 elements or that kind of area. It just has so little overhead besides creating the extra arrays. That’s hard to compete with. I mean, it’s super optimized. But Lo-Dash has also been around for a while, it’s on its second major version and I think he’s currently working on the third major version.  So, I sort of feel it’s also got that going forward where they have a lot of time to really optimize it. AJ:   I'm just going to troll a little bit. What I'm hearing you say is that if I have a choice for four items and I want to sort using either your library or his library, it’s really okay if I use BogoSort. DANIEL:   [Chuckles] Well, for four items, BogoSort, it could probably get away with it. It’s not going to last you much higher than that. AJ:   It gets up to about seven before it gets really slow. [Laughter] DANIEL:   Highly practical. I bet there's a BogoSort.js out there. AJ:   I did some Perf test on BogoSort because I want to know at what point one person’s library is better than another. Anyway, sorry. That was a sidetrack. DANIEL:   Well, it needed to be done. [Chuckles] CHUCK:   Yeah. DANIEL:   Yeah. CHUCK:   So, what I'm hearing though is that Lo-Dash is highly optimized and does very well on the performance. And Lazy can save you in the areas where there's work not performed or on work not preformed, that’s a savings because it’s just not doing that work at all. If you had to iterate over all of the elements in like a thousand element array, and you had to do it on every transition using Lo-Dash and Lazy.js, how do they compare there? DANIEL:   In my experience and I feel bad talking smack when John’s not here, but I do think Lazy is going to be faster when you have large collections like that. And the reason is, and I actually kind of -- this occurred to me the other day and I got a kick out of it because I'm a nerd. But I was thinking about the name Lazy and of course, I gave it that name because of lazy evaluation. But what I think is also important to realize is that it’s also lazy in the sense of doing less work. So like, in the Underscore/Lo-Dash world, if you call map on an array of a thousand elements, that gives you a new array of a thousand new elements. And then, if you want to iterate over that, then you can iterate over that. But if you think about, if you just ignore the existence of any libraries and you were just writing raw JavaScript to do the same thing, you probably wouldn’t bother creating a new array. You would just iterate over the original array and do something with the results of whatever your mapping function is. So for example, if you have a thousand people and you call map person.lastname, and then you iterate over those last names, you wouldn’t really do that if you were writing just procedural code. You would just iterate over the people and do something with each person’s last name, if that makes sense. So, if you did it that way, you wouldn’t be creating this whole extra thousand element array, right? Lazy actually is closer to that than it is to Lo-Dash because even though you create this sequence object, when you actually iterate, it’s not creating a new array and populating it. It’s just iterating over the original array and applying the mapping function to each element. So, that big difference in the amount of work being done does become evident with large collections. And in my experience, and I've been meaning to sort of like discuss this in more depth on my blog, but when you get arrays that large, collections that large, Lazy definitely does beat the competition even just for something as simple as calling map and then calling each on the result because it’s not creating that whole extra array and adding elements to it. CHUCK:   So, have you found any cases where Lo-Dash or Underscore are more performant than Lazy? DANIEL:   Yeah. Like I said on the really small collections, I actually was a little bit saddened recently because up until recently, all my tests showed that Lazy was faster than Lo-Dash for ten elements or up on almost everything. But Lo-Dash, I think, it recently has gotten to where it’s faster on ten elements. But I think it’s like around 20 or 40 now that Lazy becomes faster mostly across the board. I should also clarify that I don’t mean this as it is at all. But pretty much for almost everything, both Lazy and Lo-Dash are way faster than Underscore. I think Underscore, as a library, I don’t know the maintainer personally, but my understanding is that it sort of values simplicity and clarity as a library. And it’s true. If you look at the source for Underscore, it’s a lot easier to read and sort of more straightforward, and you might argue more elegant. But it’s not as performant by a longshot as Lo-Dash or Lazy. So, if you compare them on map filter, just standard stuff, Lazy and Lo-Dash pretty much always beat Underscore by a pretty wide margin. But then compared to each other, they tend to be more neck and neck. CHUCK:  I really like that you brought up the different values for the different systems mainly because if you’re using Underscore and it’s fast enough, then it really doesn’t matter if you’re using Underscore, Lo-Dash or Lazy. And so then if you value the hackability, I guess of Underscore and like I said, it’s good enough, then great. But if you’re dealing with these large collections and you find that Underscore is holding you up at that point, then you can move over to one of these other implementations. And it’s nice that they’re out there and pushing each other along. That was one thing I really like when we talked to John is he’s like, “Yeah, I love competing.” But he does it in the spirit of kind of pushing things ahead. And it’s just cool. So yeah, keep making him work. [Chuckles] DANIEL:  [Chuckles] Oh yeah, I intend to. In fact, people who are following Lo-Dash are aware that Lo-Dash is planning on incorporating lazy evaluation into the next major version. So, it’s on the roadmap for Lo-Dash 3.0 is add lazy evaluation. I think specifically to when you’re chaining methods together. So in Lo-Dash, if you call map filter compact or whatever and then when you call .value at the end is when the plan is for Lo-Dash to only then iterate over the original sequence and do all that work. And he even mentions Lazy in the issue that he created to track the progress on that. He said this library shows that this can be done in a performant way. So yeah, I do agree. I think Underscore is maybe a little bit more of an island where it’s sort of is opinionated in its own way. But I don’t know that Underscore is paying much attention to libraries like Lazy or Lo-Dash. I could be wrong. But definitely, I’ve talked to John-David Dalton just on IM a bunch of times since starting work on Lazy and it’s been good. He’s issued words of warning when he feels like I’m going down a bad path or he’s asked me how I’ve done certain things. So, I really like competition when it’s all in good fun. I would hate the idea of me wanting to beat Lo-Dash just as a matter of because I want all the attention. I think it’s better to focus on what’s the best way we can do this? And if two libraries have a totally different approach, then it’s fun to push those two different approaches as far as they can go. CHUCK:  Now, you mentioned that Lo-Dash is looking at adding some lazy evaluation to their APIs. Have you considered actually going over there and helping them with that and either deprecating Lazy or at least lending expertise since you’ve already done this to them? DANIEL:  Yeah, I’ve certainly offered to sort help on that front. But I think there are a couple of things that make it. Well for one, I still do like the competition aspect. I think that as long as Lazy and Lo-Dash are both around, if we both have that competitive drive, I think that’s actually good. It’s actually better to me than trying to fold one library into the other. Because then even though collaboration is great too, I think the competitiveness has really driven, at least as far as I'm concerned, it made Lazy a lot better. And I think that Lo-Dash has gotten better because of it too. So, I like to keep the competition there, for one. For another thing, the paradigm of Lo-Dash is fundamentally different. So, even though he’s looking to add lazy evaluation, it’s still going to be an array goes in and then you can lazily evaluate all of these functions. But then at the end of the day, an array comes out. And that makes it a little smaller in scope than what Lazy can do. Because like I was mentioning before, event sequences, for example, or iterating over streams is another thing Lazy can do if you’re in Node. You can actually lazily evaluate a file and map reduce on the lines of the file. So, things like that are things that aren’t going to happen in Lo-Dash unless it fundamentally changes the whole paradigm, which I don’t think it should. Because I think a big part of the appeal of Lo-Dash and the big part of the appeal of Underscore is that they’re very intuitive. You’re always getting an array back. And so yeah, I’ve played around with how you would implement lazy evaluation in Lo-Dash. And it can definitely be done, but I think it’s going to be done a very different way from the way it works in Lazy. CHUCK:  So, one other thing. I’m kind of looking at the website here. I clicked on the spec results and there’s a failure on there. DANIEL:  [Chuckles] Yeah. I’m not too shocked by that. Expected 7 to be 6. That’s funny. Oh, actually, yeah. The thing about the specs -- well, actually about, I guess, the homepage in general is for one, I’m really bad about keeping that up to date. My guess is, in fact actually I know this for sure, that the specs are passing if you run them, like if you clone Lazy locally and run the specs on your machine. The problem is I used to have this whole system in the Lazy repo for generating the documentation and creating the website. And it was a manual thing where whenever I had a major update, I would update everything and push it to GitHub pages which is where the site’s hosted. And so, [chuckles] I guess I haven’t done that as recently as I should have. And so, while it’s hosting the current version of Lazy, it’s hosting a slightly older version of the specs. There’s actually another project that I’m working on. I don’t want to take up much time talking about it, but actually I started working on a new project that the whole purpose of which is to do a lot of that website documentation generation stuff because I didn’t think it belonged in Lazy. It had all this logic in the actual Lazy repo for generating the website. And I felt like that was orthogonal to the whole purpose of Lazy. So, this other project I’m working on is sort of a project to do all that stuff and it’s called autodoc. So, when I get that further along, I’ll start using that to more regularly update the Lazy website. But thanks for pointing out the spec. After this podcast, I’m definitely going to go fix, update the site. CHUCK:  Oh, there you go. Make a liar out of me. People are going to stop and go look and, “There’s no failing spec.” DANIEL:  Well, no. If history repeats itself, they’ll probably find a different failing spec by the time they check. CHUCK:  [Laughs] Right. Well, what are you using to test it? Is it Jasmine or QUnit or something else? DANIEL:  It’s Jasmine. Actually, it’s a combination of Jasmine and that new project I just mentioned, autodoc. Something that’s really, really cool that I like a lot about autodoc is actually part of the way it works is that instead of writing all these specs in Jasmine or Mocha or whatever, you can still do that. But you can also just where you’re writing your code, write some examples right above the function. And if you just follow a certain syntax for writing the examples, then autodoc can actually generate tests from your comments. And what I like about that is that a lot of projects, like Lo-Dash included, if you look at the source, above every function, it’ll have an explanation of how the function works and some example like inputs and outputs. And it’s always bothered me that that information is there but it’s in the form of a comment that’s not actually being executed or tested. So, that’s a big part of what autodoc does is it takes those examples that you’re providing for the reader and makes sure that they actually work the way you say they do. So yeah, Lazy at this point, I’ve refactored a lot of the testing for Lazy to use that format. So, it’s about half Jasmine, half autodoc at this point. CHUCK:  Oh, that’s kind of cool. So, one other question I have for you is, Lo-Dash and Lazy are kind of based around somewhat loosely in the case of Lo-Dash, depending on which APIs you’re using, they’re based around the APIs from Underscore. Have you deviated much from the Underscore APIs? DANIEL:  Yeah. Well, not too much. I think I’ve sort of followed in Lo-Dash’s footsteps on this one because Lo-Dash sort of has all of the stuff that Underscore has and then it has a bunch of other stuff on top of that. And that’s the way Lazy works too. It implements all of the map, filter, where, compact, unique shuffle, all that stuff, all those methods of Underscore. But then there’s a lot of additional things that Lazy offers. As far as I know, I’ve tried to make it consistent where it actually is implementing stuff that you can find in Underscore. But then there’s a lot of additional stuff that just had no counterpart in Underscore. Like for example, Lazy has a lot of stuff to do with strings where you can split a string and instead of creating a big array with all of the substrings, it again creates a sequence that you can iterate over and map reduce over and stuff like that. CHUCK:  That’s interesting. DANIEL:  Yeah. That’s actually another thing that I’ve always liked is if you ever go on Stack Overflow or something and you’re like, “How do I read the lines of a string in JavaScript?” Nine times out of ten, someone will be like, “Oh, split it on the newline character,” and that’ll give you a giant array. Like if you’re reading a text file, that could give you an array with ten thousand lines in it. Whereas with Lazy, you could call that same method that looks just like that. You split on a newline but then say you do take five and it’ll only read the first five lines and then you don’t have this giant array sitting around. CHUCK:  Yeah, that makes sense. AJ:  Saw that. I think that’s really cool. So, my iTunes library is 75 megabytes of XML and I was in the middle of writing a parser for it so that I can do fun stuff. And I kind of stopped because it was difficult [chuckles]. And I think I might have to look into this when I go back to it to see if it doesn’t help me out because it is definitely a problem if you’re trying to read in 70 megabytes of XML and parse it. It doesn’t work. DANIEL:  Yeah. So, that brings me to a general point about Lazy that I really like and a big part of the reason I continue working on the library. I think a lot of times, people who write parsers and things like that, run into this for sure where it’s like, say you’re writing an XML parser but say the XML is like you said, 75 megs of XML. You want to offer it in a way that it doesn’t require you to read the whole thing into memory, parse the entire thing, and then give it all back in some data structure. I think what a lot of parsers will do is they’ll offer some sort of callback mechanism. So, it’ll be like, “Parse this giant blob and then call this function for every element that you read out of it.” And that’s fine and that’s better for performance, but I think that the downside of that is that doesn’t really translate very easily to a lot of the functional constructs that we’re all very familiar with and that we all find very useful. So, it’s like either you have to pick being able to map and filter over something but then that incurs this giant overhead of having a huge array or you go with this callback method. And Lazy, it kind of combines the two. So, it offers the efficiency of having a callback strategy where you don’t have to read everything at once, but also offers the convenience and the familiarity of this sort of more Underscore API with the maps and filters and all that. So, that’s something that I think is really important. And like I talked about earlier, Lazy has this lazy JSON parsing ability. It might be something that could be done further down where I could also do some sort of lazy XML parsing. But my guess is you probably have something like that already, if you’re using a different language. AJ:  I was using JavaScript. DANIEL:  Then perfect. AJ:  Yeah, it’s Node. I do everything in JavaScript. It’s probably wrong, but I just do it because I love it. DANIEL:  [Chuckles] AJ:  And it just rolls off of my fingers. If I wasn’t using JavaScript, I’d probably use Go because that also looks really awesome and really simple. But I love JS. DANIEL:  That’s Atwood’s law, right? If it can be written in JavaScript, it will be. CHUCK:  [Laughs] I still think that’s so funny. DANIEL:  [Chuckles] It’s funny and it’s also shockingly true. There are NES emulators written in JavaScript. AJ:  Yes. DANIEL:  There’s a JVM in JavaScript. CHUCK:  That’s crazy. AJ:  Yeah. And if it can’t be written in JavaScript, it will be transpiled. DANIEL:  [Chuckles] Right. CHUCK:  So, one other thing I wanted to ask about was I keep seeing stuff about asynchronous iteration here. I don’t completely understand why you would want to do anything asynchronously with an array. I guess you could raise an event or something on each element? AJ:  Oh, oh, can I answer this one? Can I answer this one? DANIEL:  I would like AJ to answer this one. [Chuckles] AJ:  Oh, thanks. CHUCK:  Break my heart, AJ. Break it. AJ:  Pick me, pick me. Okay. So, token example, you have a bunch of web requests you want to make. So say, you download a list of users or a list of files and then you want to go get the metadata about those users or about those files. CHUCK:  Okay. AJ:  So, you’re not going to want to throw all of those into XHR requests at once. Or maybe you do, but probably not because the browser can only handle four at a time anyway. So, you want to go over them one at a time. Does that make sense? CHUCK:  Yup. So, do you tell it just to wait between them? Is that the point then of running them asynchronously? AJ:  Well, you want to wait until the one gets back or you want to do a couple at a time. CHUCK:  Oh, that makes sense because we’re doing lazy evaluation. So, it’s not, “Go get everything.” It’s, “Well, I only care about the first five, so go take five,” or whatever. DANIEL:  Okay. So, I’m going to have to jump in here because what AJ said is absolutely right but it’s also not what asynchronous iteration is in Lazy. CHUCK:  [Laughs] DANIEL:  And this has caused some confusion. You’re not the only one. I actually have an issue that somebody opened months ago that I still have to address. And it’s like the website. I’m terrible at being on top of some of this stuff. But what AJ described is one form of async iteration. And I think that’s led to some confusion because that’s probably what a lot of JavaScript devs think of as async iteration. With Lazy, when you talk about async sequences in Lazy, it might be actually what you were thinking, Chuck, which is say you have an array, the idea is rather than iterate over the entire array right now, I want to iterate over it asynchronously. So, I’ll do something for the first element right now and then on the next turn of the event loop, I’ll do something with the next element, and so forth. And I could see. I get your question. A lot of people have asked me actually, “What’s the point of that?” And for one thing, I don’t know that I know all of the use cases. But the one that I know from my experience is having worked on some fronted UIs before where dealing with large collections could cause the UI to freeze up, what’s great about asynchronous iteration is you could potentially do a lot of work, iterate over 10,000 things. But you won’t be freezing up the UI because you’ll only be looking at either one thing at a time or however you want to chunk it up. You could look at a hundred elements at a time. And that way, you could kick off an iteration that say will look at every item of 10,000 and add something to the DOM for each one. But instead of doing that all at once and then having the UI freeze up and suddenly the DOM has 10,000 elements in it, you start doing it asynchronously and then one by one, the DOM gets updated with a bunch of new elements. So, that’s one use case. And another one that I think is really good that I’ve used Lazy for actually on UIs is when you’re doing some sort of search or filter based on the user typed something into a text box. So, say the user types into a textbox and then you want to search and display all the matches from some collection but say the collection is 10,000 elements. Then rather than search through the whole collection all at once and then display all the results, maybe you just start asynchronously iterating and as you find results, you pop them all into the UI. So that’s another case where iterating asynchronously allows you to have a responsive UI and still do this pretty massive iteration, if that makes sense. But I think I might need to come up with a better name for it, because I think the explanation AJ gave is totally what most devs think I mean by asynchronous iteration. And I’ve got to think of a better way of expressing that. AJ:  And if you happen to be looking for that kind of async iteration, I posted the link to forEachAsync, Lateral, and Join in the show notes. CHUCK:  Cool. So, is there anything that Lazy does that we haven’t talked about yet? DANIEL:  No, I think I’ve touched on pretty much everything. Actually, something that just got added today, somebody posted, “Oh, I would like to see if Lazy could provide a sequence interface over the concept of every time I set a property on an object.” So, I actually just implemented a very basic version of this where you can call, you can wrap an object with Lazy, and then call watch and specify a property name. And what it’ll give back is a sequence consisting of all of the values that are assigned to that property. So, then you could call map filter over that and each or whatever. And when you call each, that callback will basically be given the result of that sequence anytime the object’s property is set. So, that’s definitely an example of reactive programming. It’s maybe not that clear to a lot of people what I’m saying so let me actually find a link. Because on the issue, I commented and provided a little illustration of what I meant, so I’ll include that here. But yeah, it provides the same sequence interface, like I said, that you can map over and so forth. But it applies to all the values that a property has over the lifespan of an object. CHUCK:  Right. DANIEL:  So, the weird non-array-based paradigm of Lazy lends itself to some of these other maybe exotic use cases. But I don’t know how useful that’s ultimately going to be. It might be or might not be. But I like that it has that flexibility that it can lend itself to these unusual uses and go places that Underscore and Lo-Dash can’t go. CHUCK:  That is really cool. Alright. Well, we’re getting toward the end of the time. I don’t want to go over. So, we’ll go ahead and start wrapping up. I’m going to make AJ start off with our picks. AJ:  Sweet. First, I’m going to pick, there’s this really awesome song that I found on Spotify. I’ll probably have to actually open the link to find the name. I think it’s called HappiNESs actually. And it’s by some retro 8-bit band that does retro game music. And I think it’s what happens if Owl City fell in love with Zelda and had babies. So, it made me happy. And if you like video game music then you might like to listen to it. And I’m also going to pick 15 things altogether, but the core of it is the AVerMedia game recorder. I’ve started using that to record the UtahJS meetings. And after much trial and error, I’ve come up with a combination of a couple of different adapters that all fit into a shoebox. And one HDMI goes out to the projector, one HDMI goes to the computer, and then the AVerMedia capture recorder sits on top with a couple of adapters dangling on the inside of the shoebox. And it’s pretty much a turn the mic on, hit the red button, and then when the presentation’s over, hit the red button and turn the mic off type system. So, it only costs about between $300 and $400 to put together, which if you’re looking at other capture solutions for conferences or for buying a complete put-together system, it’s $2000 for the next best thing I could find that had everything together. So, I’ll put a link to that. And if you have a meet-up group and you guys have a budget of $300 that you could put into a presentation recorder, I’ve got a selection of equipment that you can use to do that. CHUCK:  Alright, very cool. So, I’ll just in with a couple of picks. First pick is I’ve been listening to books on Audible. I got a little bit burned out so I started listening to a fiction book and it is called ‘New Spring’. It is the prequel to the Wheel of Time series of books. And some of those books get a little bit long in the tooth I think. But I don’t remember if Brandon Sanderson finished the series or not, but if not, then he’s pretty close. And anyway, so now, I’m going to just go and listen to all of them on Audible. So anyway, I’m really enjoying that. I’ll put a link to the book in the show notes and I’ll put a link to Audible in the show notes as well. And then that’s pretty much all I have. I just want to remind you though, if you’re thinking about going freelance or you are freelance and you have questions, I am going to be doing that Q&A. And I’ll put the link to that in the show notes as well. Dan, what are your picks? DANIEL:  So I’ve got a couple of picks that are not really new things at all. Well, the first one is kind of new to me because I just read it. It’s the book ‘Surely You’re Joking, Mr. Feynman!’ It’s by Richard Feynman who’s a pretty famous physicist. But it’s essentially a memoir, I guess, or more like a collection of little mini-memoirs. But it’s just a really awesome read if you’re into science and scientific thinking, and outside the box thinking. And yeah, I really enjoyed it. And it actually really got me thinking about some of the things, like the way to approach ideas and thinking outside the box like I said. My next pick would have to be, so there’s a band that I really like that I feel like are woefully underappreciated. They’re called Starflyer 59. And I like them a lot. They’ve been around for quite a while. But they’re not really that well-known. But when I introduce people to them, I fear that they’re not going to have a good first exposure to them. So, I specifically want to recommend their album called ‘Leave Here a Stranger’ which to me was just a great album. It came out over ten years ago, I think. So, it’s kind of an oldie. But yeah, if you want to check out a new band, they have an airy ethereal sort of sound, I would highly check them out or I would highly recommend them. And then my last pick, for something a little more recent, this is maybe an odd think to pick but there’s a website called Gittip. I thought it was pronounced git tip, but I recently discovered it’s pronounced gittip. And it’s for open source developers to basically accept donations just to keep doing what they’re doing. I’m not on it but I know a lot of devs are. And it’s pretty new. And DHH, the guy who made Rails, recently sort of wrote a blog post about mixing money and open source and how that’s a really bad idea. And that led to a conversation between the guy who founded Gittip and DHH which I found really interesting. So, I’m posting a link to that. They had a phone call over Google Hangouts which you can watch. And it’s just an interesting discussion which I consider to be still ongoing. It’s not clear whether mixing money and open source can work in the way that Gittip wants it to or not. But it’s clear that there are good points on either side and that there’s level heads on both sides. So, to me, it’s like an interesting experiment and definitely something to check out especially if you’re involved in open source and have considered, “How could I ever make a living from this?” CHUCK:  Yeah, I’m on there. [Laughs] DANIEL:  Oh, nice. [Chuckles] And you should support Chuck. AJ:  I am also on there, but I’ve never got a penny from it. CHUCK:  It looks like I’ve received 25 cents. So, I’m rich now. AJ:  Man, when I get there, I’m going to be pretty happy. DANIEL:  [Chuckles] Everybody’s got to start somewhere. Even the founder, he makes 400 a week or something. So, it’s not going to replace your fulltime job but it’s an interesting idea. And I like that they had that conversation because I think it started out as rants back and forth on Twitter where they were calling each other idiots but quickly got to be more, “You know what? Maybe we’re both reasonable people and we can have an honest conversation about this.” So… [Chuckle] CHUCK:  Yeah, alright, cool. Well, thanks for sharing and thanks for coming on the show and talking about this. It’s been an interesting way to look at programming. DANIEL:  Yeah, thanks so much for having me.  I apologize to anybody listening if I didn’t explain the ideas that well. But you can always contact me on GitHub or email me or whatever and be like, “I have no idea what this is. Can you explain it better?” aAd I’ll try to explain it better. CHUCK:  Alright, cool. Well, thank you very much. We’ll catch everybody next week.

Sign up for the Newsletter

Join our newsletter and get updates in your inbox. We won’t spam you and we respect your privacy.