047 JSJ Specialized vs Monolithic with James Halliday and Tom Dale

Download MP3



01:52 - James Halliday Introduction


Backbone.js 38:19 - Module Systems

CommonJS 41:14 - Cloud9 Use Case 43:54 - Bugs


Next Week

Why Javascript is Hard


JAMISON:  You can curse but we will just edit it out and replace it with fart noises. TOM:  I’ll be providing plenty of my own. [Laughter] JAMISON:   Okay, good. [Hosting and bandwidth provided by the Blue Box Group. Check them out at Bluebox.net.] [This episode is sponsored by Component One, makers of Wijmo. If you need stunning UI elements or awesome graphs and charts, then go to Wijmo.com and check them out.] CHUCK:  Hey everybody and welcome to Episode 47 of the JavaScript Jabber show. This week on our panel, we have AJ O’Neal. AJ:  Yo! Yo! Yo! Coming at you not even live! CHUCK:  [Laughs] Alright, Jamison Dance. JAMISON:  Hi guys, it’s tough to follow that. CHUCK:  Merrick Christensen. MERRICK:   Hey. CHUCK:   Joe Eames. JOE:   Howdy! CHUCK:   Tim Caswell. TIM:   Hello. CHUCK:   I’m Charles Max Wood from DevChat.tv. And this week, we have two guests. The first one is Tom Dale. TOM:   Hey, thanks for having me. CHUCK:   The other is James Halliday. JAMES:   Yep. Hello. CHUCK:   Welcome to the show, guys. We were having a conversation a while back, I don’t remember if it was during another episode or after another episode. But we were having a discussion over code complexity and having like small simple libraries or small simple sets of functionality versus large monolithic sets of functionality, and how to approach those and when they’re appropriate. So, we brought you guys on to help us explore this because you're experts, right? TOM:  I don’t think that’s a fair analysis of the situation, but we can certainly fumble our way through something. [Laughter] CHUCK:  Alright. So, why don’t you guys, real quick, just kind of introduce yourselves? Give us a little background on what your experience is so that we know which questions to ask you guys. James, why don’t you start? I know you’ve been on the show before. JAMES:   Hello. I suppose I wrote Browserify which is relevant here. It’s a common JS style, bundler packager thing that just uses NPM. And I have a bunch of other libraries. And I really like doing data development as just a bunch of little modules put together. They are all published completely independently on NPM. I think I’m up to like 230-ish some odd modules on NPM now. So, I’ve been doing that and I really like that style. MERRICK :  Rock on! CHUCK:  Yep, and then we also have Tom. Tom, do you want to introduce yourself? TOM:  Yes, I’d love to. So, I guess I should say, I should give it a caveat that I am not a classically trained engineer. I don’t have a degree in Computer Science or anything like that. I basically, like I said, fumbled my way through the whole thing. But perhaps, one of the most influential aspects of stuff that I worked on that has influenced how I perceive programming is I was working on the iCloud apps, the web applications for iCloud.com. At the time when I was working there, it was still called MobileMe. These are really big applications. So, they are basically the equivalent of like iCal on the desktop and the Address Book on the Mac Desktop, et cetera. And I left. And since then, probably what I’m best known for is I’ve been working on a JavaScript MVC Framework called Ember.js. Yehuda and I have a company together and we basically work on it and that’s basically my main focus right now. And I think I agree with James. I also like writing small modules that work well together. I think the difference between us, our philosophies, is that when I present it to the world, I like to present it as an integrated package that’s easy to get started with, it’s obvious where to get started with. And if you need to swap out the modules, you can do that but that’s not the default behavior, basically. JAMISON:  I just want to correct one thing that you said. You said the thing you’re best known for is Ember. I thought the thing that you were best known for was Big Data and Hadoop. TOM:  Well, Hadoop specifically, yeah. JAMISON:   And then Ember is really an extension of your work with Hadoop. TOM:  Well, Ember.js is really the premier framework for integrating with Big Data and Hadoop. Although as a side note, you know I found something out? Did you guys know that Hadoop is not real time? AJ:   Oh, my gosh! TOM:   I thought to be Big Data, you had to be real time. But, apparently not. AJ:  They go hand in hand everybody knows that. TOM:  I’m actually working on submitting a talk to Hadoop Summit. I don’t even know anything about, I don’t actually know what Hadoop does or what it is. But I think there’s an event there whereby I can talk about JavaScript and Hadoop. So, I’ve been doing some research. [Laughter] CHUCK:  Nice. Alright. So, I want to kind of get into this a little bit. It seems like a lot of people, when they talk about the trade-offs between a large set of tools like Ember versus the little one off tools that James is talking about, it seems like a lot of people tend to pick one or the other based on, will this, like you said, this gives me everything it needs to get started versus other people saying, “You know what? I just want a whole bunch of tools that do one thing and do it real well.” So, what is the trade off? I mean, when is one appropriate over the other? TOM:   Well, I don’t know. There are actually two sides of the same coin. I don’t know. I guess my approach to this is that you want to choose a solution that fits the complexity of the problem. So you want simple solutions, but you don’t want simplistic solutions. And I guess I’ve written plenty of micro libraries. In fact, if you go onto the GitHub page for my company Tilde, GitHub.com/TildeIO, you’ll actually see that we have published many different micro libraries that solve one very specific problem. And in fact, a lot of the pieces of Ember are built in terms of these, I guess what you call, micro libraries. So personally, I think that if you’re trying to solve a problem, if you can solve it using a smaller solution then that’s great, right? But usually after you think through, especially these very complex problems that we’re trying to solve, you need a bigger and more integrated solution. You should be pragmatic. You should pick the approach that works for the particular problem. I don’t have anything against micro libraries or small modules. However, we have a name for the thing which is -- we have a name for when you see the same solution to every problem. And the name for that is ideology. And I don’t really subscribe to the ideology that everything must be small, that everything must be these tiny little things. And it’s your responsibility to package those together. AJ:  Sure, sure. JAMISON:  How would you describe your approach to solving problems? AJ:  I was going to say I think that is kind of similar to the argument as to like whether a student should learn C first versus learning Python first. For a lot of people, what’s going to be more motivating is being able to see the work done, being able to see the accomplishment of something. And so for that, I would say the large frameworks make a lot more sense to come into it, right? Because you don’t want to come into all the crap of JavaScript. You want to come into a narrow channel that’s well-tuned, that helps you get stuff done and then learn more about the crap and the hard parts and the bad parts and go -- like my thought would be to come in with something like a framework and then diverting to the smaller modules and being nitpicky afterwards. TOM:  AJ, that’s a great point. And I think that leads to another point which is that oftentimes when you’re learning, when you’re new to a new domain of software engineering, you don’t know what you don’t know. And it’s bad if you are learning as a web developer and you're deploying stuff to production that’s susceptible to a lot of, for example security vulnerabilities, that as an industry, we’ve already learned how to deal with. Things like CSRF attacks, right? Like maybe, you can deal with simple CSRF attacks but there are actually many exotic varieties and you’re not aware of those things. Encodings are another good example. You’re not even aware of the problem of encodings because you’ve never dealt with it. And that’s too much complexity to load upfront. So having a framework that helps you deal with those things and then once you’ve leveled up to a point where you can understand what it’s doing, then you can swap out whatever pieces you’d like. JAMES:  I think that’s a good methodology. MERRICK:  I agree with a lot of things you’re saying, Tom. My problem is, say some of these bigger frameworks, because they grow bigger and they’re more integrated, they have to make a lot more decisions in terms of what you can use with them, like it’s really hard to keep the same level of flexibility when you are using some sort of monolithic framework upfront. Monolithic is a bad word because it carries a bit of negative connotation. But for example, when I use Ember.js, I find myself getting frustrated a lot because there’s some opinions that you guys have that I don’t have. And because it’s so integrated, I can’t necessarily change out those opinions that I have. Does that make sense? TOM:   Yeah, totally. And I would say that if you have opinions that differ largely from ours, then Ember’s probably not a good fit for you, right? That’s kind of the trade-off that you get with an opinionated framework. I think DHH feels the same way about Rails which is like -- for example, if you really like configuring your application server with a bunch of XML files, you probably shouldn’t use Rails, right? So definitely, the opinions that Yehuda and I have about how you should architect these web applications are 100% baked into Ember. It’s not something like Backbone where it’s basically, we’ll give you these primitive objects and you compose those yourself to figure out how you want to best architect this. We have some strong ideas and we think that those are really good ways. But of course, you can easily fall off the happy path if you disagree. MERRICK:   There are some things I struggle with like for example using strings to trace down an object path means that you have to hang off of a name space, right? And those things, you just have to buy in like you’re saying to use the framework effectively because you end up fighting it more. And I think that’s one of the hard things. I’ve always been attracted to these more monolithic things or these larger frameworks because the development environment is so consistent and maintainable but the nerd in me has a really hard time buying into opinions that I don’t subscribe to all the time. So, I feel like I wish there was more of a middle ground because when you buy into all the micro libraries, you end up piecing together a situation that isn’t nearly as consistent and a blessing to work in.             TOM:   Right. And I think that’s kind of fundamentally the problem with micro libraries. And the job that we have is to make sure that as an integrated solution, the point of making a modular is that you should be able to swap things out, for example. So, let me give you a good example of this. I know that both James and I share opinions about AMD and that Merrick, you do not. And that’s fine. Presumably, you guys read the DHH blog post, Rails is Omakase, and it’s kind of a similar thing. I wouldn’t put it the same way that DHH put it but it’s the vague, the rough contours of the argument you were making apply to Ember. But at the same time he says, if you want to take a piece of the menu and add your own, you should be able to do that. We’ve been doing a lot of work internally as we approach Ember 1.0. The internals are geared around the idea that maybe you want to use AMD, maybe you want to use some kind of modular. The approach that Rails takes that we take is, if you want to do something that we don’t think is a good idea, at the very least, we’ll give you the hooks into the framework so that you can put together some kind of integrated package that works in the way and with the opinions that you have. MERRICK:  Yes, which is something that I think these monolithic frameworks oftentimes don’t do but they ought to do, right? Because there is certainly a middle ground. You usually see that as projects mature, right? I’m looking forward to Ember having those kinds of things because admittedly, working with AMD in Ember has gotten a ton better than it was months ago. And these bigger full stack frameworks also tend to, when they do, I think they do try to make themselves modular. I find very few that at least get to be to a good standpoint in popularity that at least don’t say they are modular, they say they are. When it comes down to it, what they anticipate that you might want to swap out, isn’t what you end up wanting to swap out. And it just comes down to the fact that you can’t be, it’s hard to be truly, truly modular where you could swap out truly anything. And like you said in the beginning, if your opinions differ way too much from the opinions in this framework, then you shouldn’t be using this framework. CHUCK:   I want to jump in here real quick. I have something to say that’s kind of along the same lines or more of a question, I guess, for Tom and James. But is it easier to write something that is tightly coupled together like that, versus making it modular? And what are the challenges in doing something like that? JAMISON:  I can jump in just from having done this a little bit, not at the scale that either of these two guys have but yes it is way harder, it’s much harder to write stuff that’s modular and allows people to hook into. It makes the code more complex, at least when I did it. Maybe it just means I’m bad at it. It looks like James is trying to talk. CHUCK:   Yes. I’m really curious to hear what he has to say because he does kind of have the experience from the other end. It looks like he’s having Skype difficulties. JAMISON:  Everybody look in the mirror and say ‘James’, or actually say ‘substack’ three times. TIM:   There you go. CHUCK:   Click your heels together three times. TIM:   Substack…substack…substack. [Laughter] CHUCK:  Alright, Tim. TIM:   Well, I was going to say earlier that coming from my perspective where I am often teaching new people how to program and getting people started in the field, that large frameworks are actually a problem. Because first you have to learn the language, you have to learn programming. For example, I had some interns last summer. And the job I got for them, the employer said, “Alright. Well, you need to use Twitter Bootstrap, you need to use MongoDB or CouchDB and you need to use all these large frameworks.” So they basically had like ten technologies to learn at once which was just completely overwhelming. TOM:   But that’s not a fair comparison because none of those are really well integrated. I think the counter example is that many people learned to be web developers using Rails without a deep understanding of Ruby. And many people, on the client side, have learned to be web developers using jQuery without a deep understanding of JavaScript. JOE:   See but this is my contention with that and maybe this isn’t a real valid contention. But the people that I’ve met that do have that approach, I wouldn’t call them web developers or server developers. They’re Rails developers and jQuery developers. They don’t understand web development. They truly just understand jQuery and without it, they’re kind of lost. CHUCK:   Yes. But the flip side is that you do kind of get in and the frameworks provide you with a pattern to follow. You kind of have this anointed path and if you set up the models a certain way, you set up your controllers a certain way, you set up your views a certain way, it’ll work and then you can start to branch out. So then, you start fiddling with the CSS or the JavaScript or whatever and you start to realize, “Okay. Well, if I really want to get the real power behind this, then I need to understand the underlying structure and the underlying language features so that when I need to something that’s a little outside of the cookie cutter that I’ve been handed, then I can do it.” And so, I think it is a good way to get in but at the same time, it really depends on the problem that you’re trying to solve because most people aren’t just learning to program just to learn to program. They usually have some problem in mind. MERRICK:  I think one of the things for me is like, it’s a lot easier for me to learn by pulling say one of sub stacks modules and looking at that source because it’s isolated and it has a single purpose and I don’t have to look at any of the integration pieces. It’s just like, it is what it is. Does that make sense? JOE:   It depends a lot on the personality of the person as well. Because like maybe the people that you’re thinking of Merrick that you need, those are people that aren’t driven by learning new things. So if you learn framework, depending on your personality, you’re potentially going to be driven to stretch out, learn new things and you’re going to want to start on CSS. And pretty soon, you’re going to want to learn something else besides Rails. MERRICK:   Yes. And maybe my expectations are too high for people but I would like it if the majority of the people that use the framework would be able to rebuild that framework. I don’t know. But obviously, that’s a really high demand but I think that’s a really good way. JOE:   I mean, think about it. Can you re-implement TCP? But you program as TCP all the time. MERRICK:   I can re-implement TCP with HTML tables. [Laughter] MERRICK:   No. That’s an excellent point. And I think I’ve probably been a little bit spoiled that way. JOE:   And it is the difference because we’ve reached a point where we can abstract with TCP. People don’t need to understand TCP to be a web developer. You might need to understand one little tiny principle, but that’s it. MERRICK:   And being able to understand how the frameworks work, to me gives you so much more power in terms of how to use the framework. JOE:  Yeah. CHUCK:   Anyway. Let’s get back to the topic a little bit. James, are you able to chime in here? I feel kind of bad because I really want to hear what he has to say. But let’s talk about a little bit more of this. It sounds like with something large like Ember, you’re talking about breaking it apart so that you can swap things in and out and you just provide kind of an API that people plug into. ‘Hooks’ is what you said, and if you’re familiar with those terms, then it makes sense. But anyway, why is it so hard to make things modular? Why is it so much work? TOM:   It’s not hard to make things modular. It’s hard to make things modular in a way that they all compose well together, from the very bottom all the way to the top. So like I said earlier, we try to extract these micro libraries from problems that we’re solving with Ember. But I think Yehuda and I spent far more time talking and discussing, then we do actually coding. We like to go for walks down the Embarcadero, which is less romantic than it sounds, and kind of really thoroughly discuss these very hard CS problems from top to bottom. What is the core of the problem? What are people trying to do? And then from that core piece, we kind of back up and say, “Okay. How can we expose that in a way that’s really approachable to people? In a way that doesn’t expose the concepts to them?” It’s better to hide things from people than to force them to learn about it and deal with it. And I think that’s why it’s taken us almost two years to write Ember because we really want to think through these problems. And I think that’s the piece that you miss if you focus too much on the modular. I think that’s good, right? You have to write modular software if you’re going to write good software. I think James and I definitely agree on that. But I believe that how those modules compose together, has to be influenced by viewing the problem holistically. And I think that’s the piece that is missing for a lot of people where we say, “Okay, just go and find all of your favorite modules from NMP and put them all together.” That piece of stepping back and looking at the holistic picture and integrating, that’s what’s missing. CHUCK:   Okay. So I kind of want to hear things from the other end. I know some of you guys do deal with things on the other end. TIM:   I know these frameworks and I’ve built many of my own frameworks yet I find myself in practice never ever using them. If I have a problem to solve, the first thing I’m going to do is spend about ten minutes seeing what’s out there. And if nothing there fits exactly what I want, I’ll just write my own. And that’s the biggest beef I have with frameworks is you have to make some decisions about what this framework is good for. Like you take Rails for example, it’s really, really good at making web applications that have a database, that have some HTML templates and do something. Like if you’re writing that kind of app, you can just throw that instantly using Rails, yet the kind of things I want to create are always unique. I want to do something that’s never been done before. And if it’s never been done before, then how is a framework going to help you do something that’s never been done before? That’s a lot harder to make frameworks that can do that. And that’s where the smaller libraries work well because well, I need an MD5 hash, has someone written that code? It’s kind of hard to do by hand and that could be a micro library. AJ:   So to your point Tim, I started Ruby with Rails and I thought it was way cool because I could do a few simple things really easily. But then when I got to the point of wanting to customize it, it was too hard. So when I got into node where there wasn’t a framework yet, it was definitely a breath of fresh air. And I was actually doing things faster with connect than I could do with Rails because of the specific things I was trying to do. So I totally agree with that. But I still feel like the experience of coming into a framework to learn some patterns and then get the general idea is much better than trying to start from scratch. It’s like learning memory management when what you want to do is add numbers. TOM:   Right. And Tim, the thing you have to keep in mind is that you’re Tim Caswell, not everyone is you. And if everyone was, the world would be a much better place. TIM:   Or we’d have thousands and thousands of modules that don’t work together, one of the two. [Laughter] TOM:   So obviously, we all only have a limited amount of time here on the planet and I spend time thinking about the things that I’m working on, are they good? Are they valuable? And I think the way that I look at these problems is in terms of leverage. So I can work on a particular problem that maybe doesn’t affect that many people. Or I can work on solving a problem that helps other people solve the problem that they want to solve. And the thing that I realized it’s just, it’s 2013, the world is powered by software. There are going to be lots of people who are writing that software that are not us. They don’t actually care about the craft. They’re not going to be doing it for the problem solving. For them, it’s a job. I think that you can -- so DHH has a thing which is that there is value in climbing the mountain together, there’s value in having the experts be on more or less the same path as the beginners. There are numerous good things that fall out of that. So for me, I want to provide a thing where people don’t have to learn all the intricacies, they don’t have to get in-depth with the framework. They can deliver something of value relatively easily, relatively quickly and get on with their lives and go home and be with their families on the weekend and not to have like focus on the intricacies of anything. CHUCK:   So my thing is that -- and I’m probably just going to summarize what Tom just said.  But there are a certain number of people that have a problem that are all fairly similar. So I’m trying to deal with something where I have this backend service that provides a ton of data that I need to organize in a certain way. And so, something like Ember gives me a lot of conventions for dealing with that and thinking about those problems. To Tim’s point, if I’m going to do something that’s totally different, that has a totally different application, Ember may not be a good fit. But if it gives me a pattern to solve a problem that I actually have, then the framework has a lot of value. MERRICK:   Like Tom said, he thinks about it from the problem as a whole and that’s awesome because when you’re solving that problem, you’re going to have a better experience than looking anywhere else. But if you want to use your same tool solving a completely different problem, then sometimes you can be fighting against it. Is that fair, Tom? TOM:   Oh yeah. I think that you guys have summed it up well. I don’t want people to get the idea that I think Ember is a good tool for everyone to use, right? There are certain classes of problems where I think it will make you more productive faster. Tim, probably you are not a good candidate to use Ember. [Laughter] JAMISON:  When you write in Crypto libraries in the browser, Ember doesn’t help you with that? TOM:   Not yet. But we have in version two maybe. TIM:  What about parsing Linux towards kernel output? CHUCK:   I remember at one point, Yehuda pointed out that the ‘to do list’ app wasn’t a terrific example of use for Ember and to me that kind of drove the point home too. It depends on the tool, it depends on the problem because if I were going to do a ‘to do app’, I probably would use Backbone just because it is so simple and there really aren’t that many moving parts. It’s when you have the big lists of objects and things that it really pays off to use something that is built to organize that. TOM:   Yeah, absolutely. I’m a big fan of picking the right tool for the job. A lot of people think that I think that all web applications should be JavaScript-rich MVC. I don’t think that’s true. I think that there are a lot of applications that are better served by a traditional Rails app, the Omakase stack, if you will. I think that there are a lot of web applications that are served by little pieces of Backbone code. It’s all about picking the right tool for the job. JOE:   Sure. I agree with that sentiment. TIM:   I have an analogy. I used to do Python GTK apps back in the day, right after I left Swing and swore I’d never do Java again. And the cool thing about GTK like most GUI frameworks is you have widgets and that’s great. You want a combobox, you got a combobox. You want a treeview, there’s that. And with the new platform of the web, you have HTML and CSS. The browser doesn’t give you all these fancy widgets. And so, I think that’s a similar situation of the micro frameworks versus the more structured system, the more structured frameworks. Do you think that’s a valid analogy? Does it make sense at all? TOM:  Well, you probably shouldn’t bring up widget frameworks because I used to work on SproutCore. That’s a little bit of a sore subject. [Laughter] TOM:   But in seriousness, I agree with you. I think that’s a good analogy. But at the same time, I think that there are people who are coming to the web that are frustrated that they want very simple controls that aren’t available. And you actually see that like you see auto-complete fields, you see things like slick grid, you see big tables, you see things like jQuery mobile and jQuery UI, where the web is clearly deficient in terms of providing everything that people need. Especially when all they want to do is slap something together and ship it and call it a day. TIM:   Right. And even more so, I’ve been working on a platform on and off for the last while that’s based on open GL, it’s basically a web GL type thing. And there you’ve got nothing. At least in HTML, you have text and divs, and CSS, and boxes, and borders. But when you’re doing GL, you’ve got shaders and vectors. And if you want to make a text widget or a dialogue box, good luck. That’s going to be a lot of work. Yeah, it’s insanely powerful. You have this very simple system with these very simple primitives and all these massive parallel GPU. So, there’s definitely trade-offs there. TOM:   I really wish that James was here to participate because I have some questions to ask him. But I actually think that there is a fair bit of analogy between Ember and the project he’s been working on most recently which is Voxel.js which is really cool. I think Voxel.js is really cool. I’ve been playing around with it. I think that it’s actually very similar to Ember in the sense that -- it’s similar to Ember other than that there’s no standard distribution. But Voxel is basically like a 3D rendering engine. There are many modules that you put together and it all works together and that’s basically what Ember is. Ember is just a standard set of those modules put together. In fact, if you go and look at the Ember source code, if you go and look at the GitHub repo, you go into the packages directory, you will see the equivalent of, it’s broken up into different packages. And each one of those packages provides a small piece of functionality. TIM:   Right. Recently, I was working with TJ on his component JS system and I want to make some UI widgets for it because I was making an app and I needed like a slider where you can resize things, I needed a treeview which basically just HTML doesn’t have. And so, I was asking TJ, “Are there any conventions in component?” He’s like, “Well, not really except for maybe your object has a .el property that’s your root element.” And so basically, I had to make a set of widgets and then make up conventions for them on the fly. And as long as you use my widgets together, you have a framework. But if I wanted to use someone else’s micro component that used different conventions, then I’d have to shim it to be able to use it. MERRICK:   Yeah. And that’s why it feels so fragmented when you’re using those kinds of things. TIM:   It’s not so much the monolithic versus the modular. It’s, are your conventions compatible? MERRICK:   Yes. JAMES:   Can you guys hear me? IN UNISON:  Yes! Yeah! TIM:   Tom was saying that Voxel.js is monolithic, I believe. JAMES:   Yeah, it is kind of monolithic. [Laughter] JAMES:   No, no, no. It’s not a good thing by any means, it certainly could be more minimal. I’m just not really sure how to do that for a 3D system. It’s not really… TOM:   Yeah. I’m also not sure how to do that well for a JavaScript MVC framework. JAMES:   Well maybe, it’s just not possible for MVC frameworks. And I don’t even use those. So, I don’t really know much. AJ:   Backbone does. TOM:  So James, I had a question for you that I wanted to ask which is, have you gone and looked at the Ember source code ever? JAMES:   No. I don’t think so. TOM:   Okay. I think it was you that posted a comment on Google+ one time that accused me of being a peddler of monolithic software. [Laughter] TOM:   I’m not angry. I’m just trolling lightly right now. But I think someone said that. JAMES:  Okay. I really don’t think I said that. I really don’t even know much of Ember in the first place. TOM:  I’m just teasing you lightly. MERRICK:   I’ll call you that, dude! You’re such a peddler. Every time I release a library, you troll me about the lines of code it has. TOM:   Confirmed. [Laughter] TOM:   So anyway James, the thing that I want to say is actually, I think that you and I actually might have a more similar world view than people might think because if you actually go and look at the Ember source code on GitHub, there’s a directory called Packages. And inside of Packages, there are a number of packages and each one of those is an individual piece that provides some value to the system. JAMES:   Okay. So, why aren’t those separate projects though? Because this is one thing that I think is really important for splitting things up properly is to split them up entirely so that it’s really easy to contribute, so that you get independent versioning and so that each piece is completely independent and can be reused by other systems because what’s the good of reuse if we can’t actually reuse the pieces trivially? Like, reach in and rip them out. TOM:   So, there’s two answers to that. The first one is that many of the dependencies of Ember, we actually have released as micro libraries which are totally separate projects. You can use them with Backbone. You can use them with whatever. Usually when we release these micro libraries, we try to make them run in both node and in the browser which is probably more painful than an iron maiden. But the real answer to the question, why aren’t these provided as separate things is because there is no good package manager for the web. And in fact, we were distributing as separate packages back two years ago, we were working on a project called BPM, the Browser Package Manager, which unfortunately we haven’t had the resources to work on again yet. But the intention absolutely was to release these as separate projects. There’s just no good infrastructure for doing that for the browser right now. JAMISON:   I disagree. NPM is for JavaScript and it works. And most importantly of all, it has, I think, it has over 20,000 packages right now. I mean… TOM:   You just highlighted why it doesn’t work. JAMISON:   Why it doesn’t work? TOM:  Because there are 20,000 packages. And when I am… [Crosstalk] JAMISON:  Okay, so… [Crosstalk] TOM:   …getting started and I have to find jQuery for the browser, is it jQuery, lower case? Is it jQuery, capital Q? Is it browser dash jQuery. I have no clue. TIM:  That was my fault. [Crosstalk] JAMISON:  So, that is good. Don’t use that in the first place. Don’t use jQuery, if you’re going to do modules that’s really different from -- I mean, jQuery was built in the age before modules. So, it has a lot of relics for that. If you’re going to use jQuery, just throw it in the script tag. It’s really not meant for the future, the modular future that we’re racing towards. TOM:   Have you read the source code for jQuery 2.0? JAMISON:  Oh, it’s so bad. I actually have read the jQuery, well not 2.0. It’s like 1.-something. It’s so bad. Anyway, who cares? So, the so-called problem that you’ve mentioned is not actually a problem. This is just an unavoidable thing that happens when something is useful and popular. You have 20,000 packages and that’s good. People make exactly the same complaint about CPAN or about Ruby Gems. It’s a good problem to have, I think. Because the alternative is, sure you might be able to find jQuery or Backbone or some popular packages, but you can’t find anything else since the people don’t write modular programs with rich dependency graphs because there just isn’t anything to depend on and that is much, much worse. TOM:   No. So, there is a difference between node and the browser. The difference is that if I’m using five different packages and each of them have five different dependencies, each of them uses a different library for parsing JSON or whatever, or parsing XML. Then on the node, it doesn’t matter because I can just load it in memory, not a big deal. In the browser, if I have several different dependencies and they all depend on different versions of parsers, they all depend on different versions of jQuery, I have to load those over the wire. MERRICK:   I don’t think it’s so simple to say that it’s “just” JavaScript. CHUCK:   Yeah. We are getting a little off topic, though. AJ:   I think this is the exact topic. CHUCK:   Okay. AJ:   And I tend to agree, not knowing whether or not when you NPM install something if it’s going to be browser compatible. And I may have to potentially have to run Browserify to make it browser compatible. But even then, it’s still no guarantee. That doesn’t really seem like a solid experience. JAMES:   But actually, there is a solid way to check this now. I’ve been working on this project called Testling CI. So, if you go to Browserify.org/search and you do a search on that, you’ll get, at the top of the results, kind of like how search engines have ads a little bit. But if something has been run through Testling CI, that will show up at the top of the results with exactly which browsers that module works on. TOM:   There’s a thing I want you address. The thing that I’d like you to address is how do you deal with the fact that if you have many, many, many different solutions to similar problems, and there’s a culture of, “If it doesn’t do exactly what I want, let me just rewrite it and write my own version,” and you have many widespread dependencies. In the browser, you have to transfer that every time the user loads the page. And as far as I know, and please do correct me if I’m wrong, there’s no equivalent to something like bundler in NPM that will try to distill all those dependencies, all different versions of a dependency down to a single version that they could share in common. JAMISON:  NPM actually does this by default if the versions are compatible. If you have, say I use the library through and two packages both have a compatible range, and so, NPM will actually move that up a level. And it does this recursively for all of the module levels. Sure, you might get version discrepancies. You probably will, if you have really a lot of dependencies. But I think that that’s actually a feature because for me personally, it’s much, much more important that these pieces work in the first place than if they are exactly optimized. Most of these libraries are very tiny in the first place. I mean like, maybe ten kilobytes is a big module, and then you can minify and GZip it and it’s nothing at that point. So, I don’t know. People make this complaint a lot but I really haven’t seen it so much. TOM:  So, the reason I think it’s an issue is because I see many web applications in the wild that are built on small frameworks like Backbone that end up being like 500, 600, 700, or 800, 900 K. So, it seems obvious that if you -- something’s happening. One is either web developers are having to write a lot of code themselves to build their web application or they’re having to include many, many dependencies that don’t know about each other. And because there’s no shared ecosystem, there’s no kind of standard library, if you will. They each have to implement their own -- for example foreach. If you’re using something like component, then each library can’t depend on the fact that foreach has been shimmed in something like Internet Explorer. So, it has to shim foreach. Even though if you knew that you had a shared dependency, you could just say, I’ll just use the one available there. JAMES:  Well, that’s exactly where explicit dependencies in the pack of JSON can help you out and while having a separate module to do that is very beneficial. I don’t really think that it’s a problem with micro frameworks so much as a problem not having a module system or not using a module system. Once you use a module system, you can actually explicitly say, “Okay, I’m going to depend on this package and this package can do all the work for me,” instead of having to inline everything. CHUCK:  When you’re talking about a module system, you’re talking about like AMD? JAMES:   I’m talking like Common.js. AMD would be another example that I don’t like as much. TIM:   So, if I can jump in here. The issue, what it sounds like to me, isn’t really about whether there’s a module system or not. That’s just why packaging them as one library is useful is because it’s the best way to package it. I think the real issue is how do these libraries talk to each other? In node, we have very simple conventions. There’s the node callback convention, there’s the node stream interface. And for the most part, this is pretty much all libraries need to talk to each other. This library takes in a stream out with some other stream. And so, you don’t need a whole lot of conventions. But when you get into other use cases like maybe you’re making UI widgets, then UI widgets need some conventions. How do you get to the root element? How do you tell a UI widget to redraw itself? How do you do a vent delegation? There’s just a much bigger surface area of how these modules need to interact. MERRICK:   How do they communicate to each other? Do they trigger events on themselves? Do they have some sort of known API? TIM:  Right. And if there’s no conventions, then every library is going to do it differently. So it’s not so much of whether they are bundled or not, that’s a packaging issue. The problem is whether they agree or not. TOM:  I think this comes back to thinking about the whole thing holistically and climbing the mountain together, if you will. There are a lot of trivial choices. And they’re choices that end up not really mattering in the long term, but developers love to argues. So, developers are going to argue about, for example, should I use low dash or should I use underscore? Now, if I’m building a web application where there’s many, many small dependencies and there’s not kind of a shared ecosystem where everyone just accepts, “Okay, you know what? We’re just going to use underscore.” Now, I have one dependency that uses underscore, one dependency that uses low dash, and another dependency that uses whatever else. And these things actually add up. These things add up over time. MERRICK:   Or for framing latency? TOM:   Yeah. Just in terms of the amount of JavaScript, you have to ship over the wire. Focusing on trimming that down is actually important. It’s not on node. I think NPM is great, don’t get me wrong. I think NPM is fantastic software. I think it solves the problem extraordinarily well on node. But the browser has so many more constraints than something running on the server. TIM:  It matters in node too. AJ:   But the thing is the NPM is not the problem there. The problem is that developers going in with the mindset of, not considering those problems. You can have any sort of package manager, they’re all going to have that problem until the developers are disciplined to understand it and code to that problem. TOM:  Right. And that’s why I think it’s largely a community issue. It’s largely a community ethos issue. TIM:  So, the Cloud 9 Use Case, I worked on that. And from what I understand, that was largely why ddoop was added to NPM in the first place. NPM used to have this behavior that James described where if you have two compatible dependencies, it only installs them once. And then somewhere down the line, that behavior changed where even if they’re compatible, it will still install two copies. When you have a large project like Cloud 9, we end up with a gigabyte of JavaScript just to get a server started and that’s insane. TOM:   Cloud 9 is awesome, by the way. Good job. TIM:   Cloud 9 is cool. But there’s a lot of JavaScript in there because it’s a whole lot of tiny libraries with no conventions at all. And crazy stuff, like I want Web Sockets. So, I’m going to depend on the WS library. Well, the WS library has this CLI component which depends on these five CLI libraries and I don’t know. It explodes. It’s a problem there too. I mean, it’s not as sensitive as sending JavaScript over the wire across the Internet, but it’s still a sensitive issue, just a different magnitude. TOM:  Tim, you’ve just described the problem that I’ve encountered every time I’ve tried to build a big application. CHUCK:  So, to boil it down. Really what it seems to come down to is like Tim said, with the large -- I don’t want to use the word ‘monolithic’, let’s say fully featured libraries. Everything in there is designed to work together and the one off libraries, the thing is it does their jobs well which is really nice, but they don’t always talk well together.  And so, depending on what your problem is, you may find that it’s simpler to go with something that everything is there. Some things you may not need but overall, everything works well together and you get everything you want in just one big library versus having to go figure out what all the pieces are that you’re going to do what you need to do. JAMISON:  Well, I think that’s a mischaracterization because when you have something that all of the pieces work well together, there can’t be that many pieces because the institutional overhead of managing that kind of a thing is much higher than a completely distributed ad hoc community like NPM where everybody is just doing whatever. So, the benefit of the other side of everybody’s doing whatever Tom has said is that you have a much broader swap of scope. So, it’s much more likely to have some obscure module that you need that does that one thing very well, that you just don’t get in a more integrated community. CHUCK:   I totally agree. You also get the variety, you get the… TOM:  It’s also much more likely to have exotic bugs, obscure bugs that you’ve never heard of either. Like if you look at the jQuery source code, in fact now, in jQuery 2.0, there is more -- so, jQuery 2.0 has now pulled out support for the old IE’s. There is now more code in jQuery 2.0 that deals with WebKit and Firefox bugs than IE bugs. CHUCK:  Interesting. TIM:  That doesn’t surprise me. TOM:  Right. So, people have this misconception that, “Oh, if all I target was WebKit, then I don’t need to worry about all these crazy IE work around and that will reduce the code dramatically.” In fact, no. WebKit and the various JavaScript run times have had crazy insane bugs. And you have to work around those because they are still deployed, right? You still have users who are coming and visiting your website, using those run times. So if you have different dependencies and some of them solve some of the bugs, you’re going to have a worse experience than something like jQuery where you know everyone is on it so you know it’s well tested, it’s well run through and you have an entire community, an entire organization around fixing those bugs and making sure there’s a consistent experience across all these very different browsers. JAMES:   I still don’t think that you need it packaged together. I think that we should all work together to solve the module system for the browser and I don’t know if we’ll ever agree on what the best solution is. TOM:  It’s coming in ES6. Dave Herman’s working on it. Dave Herman’s a very smart guy. You talked to him recently on the show. He will solve the problem. JAMISON:   I disagree that anything is going to happen in ES6 that anyone will like. I don’t even know that people on ES6 will like what they come up with. [Laughter] JAMES:  Well, if they agree, then that will help. TOM:  Wait, what does that even mean? JAMISON:   I’m just completely pessimistic. I’ve just completely written ES6 off at making a good module. I’m not convinced that they will be able to pull anything off that anyone likes. AJ:  But that whole system looks like Python, why didn’t they choose JavaScript, Common JS, or AMD, like it’s so stupid. Why make another standard and not standardize? JAMISON:  They’re not aware of what people on the ground have been doing all of these years. They are just inventing more language abstractions that nobody needs. CHUCK:  So now that we’re down this rabbit hole, why don’t we go ahead and do picks? [Laughter] TOM:  We can do that as long as you’ll promise me that you’ll have James and Dave Herman on for another episode to talk about this. JOE:  I was going to say James. JAMISON:  And bring Isaac, also. JAMES:  Isaac would be good, yes. CHUCK:   Yes, we‘ll see what we can do. I think it’s definitely worth discussing. I didn’t realize that there was a module system coming down in ES6. So, we can definitely talk about the pros and cons and maybe some of the things that they could consider or should have considered and see where we go with it. I think it will be really interesting. TIM:  So, my point before I mentioned modules and got sidetracked was that, ignoring that problem, then all that’s left is we just need conventions that people can agree on, whether they are packaged as one package or a thousand micro libs doesn’t matter as long as they work together. One of my favorite books by the pragmatic people was ‘Interface-Oriented Programming’. And the book basically says it’s all about your interfaces, how things work together. That’s all that really matters in the end when you have modular code. CHUCK:   Yeah, we talked to Sandi Metz on Ruby Rogues and she said a lot of the same. It’s how they interact, not what they are that really defines what your code is or what your program is. Anyway, let’s get to picks. Merrick, why don’t you start us off? MERRICK:  Sure. I’m going to pick jQuery 2.0 and also the ECMAScript 6 Module Definition. [Laughter] CHUCK:  I think that troll has horns. MERRICK:  And also AMD, because I actually think all of those things are awesome. CHUCK:  Alright. Joe, what are your picks? JOE:  I’m going to pick two things here. There’s a musical artist named Yiruma. I think he’s a Korean, composes original piano pieces and his stuff is just awesome, absolutely awesome. I’m going to pick him. And then, I’m also going to pick a TV show ‘Elementary’ which is apparently the hottest new show on TV. I’ve been watching it since the season started and it’s freaking awesome. CHUCK:  I didn’t really get into it. JOE:   Oh, man! It’s amazing. It’s because you have no taste. CHUCK:  That is very likely true. AJ, what are your picks? AJ:  So, the most amazing thing that’s happened to me perhaps in my life even was my friend had a birthday party and it was a tasting party. He bought these tablets called Miracle Berry Tablets. And Miracle Berry is a fruit that has a very unique and special protein in it called miraculin, after the word miracle. [Laughter] AJ:   When it binds to your taste buds, it makes everything terrible taste amazing. So, you can take something like grapefruit after you dissolve one of these tablets on your tongue and it tastes like the sweetest, most delicious amazing fruit ever. You just pick up nasty stuff out of the fridge that you hate and stick it in your mouth and the worst that it would normally taste, the better it tastes with the Miracle Berry Tablets. The only downside is that once you put that on your tongue and everything tastes delicious, you start putting these terrible foods in your mouth particularly the citrus fruits is what it’s really, really, really powerful for. So, just add it to lemons and limes and all that acid, still makes your tongue raw and eventually bleed but you don’t feel it until the tablet wears off. CHUCK:  Sounds like a bonus feature to me. So, did you want us to sell it and then sell it to our friends? Or try it and then sell it to our friends? [Laughter] AJ:  Yes. Oh, my gosh! This stuff’s amazing. You’ve got to have that experience just once. It’s a little expensive but no more expensive than your common drug. TIM:   Are you promoting drug abuse? Wait a minute. [Laughter] JAMES:  I also see THC on this agreement list, AJ. I think there might be something else miraculous about these miracle berries. TOM:  AJ, I want to come to your parties. AJ:   One other thing is I compiled together. Maybe I mentioned this on the last show. I don’t know but I compiled together a script for Ubuntu. So, if you’re getting started with Ubuntu, you’ve made the jump to get an ALS and you’re trying to commit yourself to it, put together a script that install all the common things for you like Skype and Chrome and Steam. All the stuff that you would expect to kind of have on it already if you had bought it preinstalled from like Dell or HP at Wal-Mart, you know. JAMISON:   Does it have the Ask.com toolbar on it because I can’t browse the web without my Ask.com toolbar. AJ:   None of that stuff. It just installs the things that you should get like the Microsoft fonts, Times New Roman, Arial, yada...yada...just the things that the DVD player, the Blue Ray player, stuff that they don’t include by default but should be there. CHUCK:  Alright. Jamison, what are your picks? JAMISON:  Mine is just one, it is a Dwarf Fortress comic called Bravemule. So if you don’t know what Dwarf Fortress is, it’s a game made by this math genius hermit guy that is basically a super realistic simulation of controlling this colony of dwarfs. It simulates their sanity and their internal organs and it’s incredibly detailed and complex and hard to get into it and I’ll never ever play it, but it creates these amazing stories. So this guy, or actually this team of people, documented one of their Dwarf Fortress games on this website called Bravemule and it’s amazing. It captures just the weirdness of the game and all the crazy things that happen and it’s got this really cool comic art style that accompanies it and stuff. Even if you’ve never played the game, aren’t interested in it at all, it’s a really good story. So, just check it out, it’s just Bravemule.com. CHUCK:   Alright. Tim, what are your picks? TIM:   Alright, I’ve got two. One of them is RealtimeConf EU, because I actually get to go this summer in France and it will be a blast. So, if any of our listeners are in Europe or any of our other people like flying to Europe and can afford it, I think it will be a really good conference. And the other is one of TJ’s latest experiments. I’m a sucker for experiments. He has a project called CPM, the C Package Manager, which is kind of like NPM or component but for C. I don’t know that sounds neat because historically, all C programmers basically just copy paste or dynamically link all their dependencies. I think it would be cool to have a package manager, JavaScript style. JAMISON:   Isn’t that a harder problem to solve in compiled languages too though? TIM:  No, the source code is pretty portable. The linker problem is a pain. How do you link binaries? I don’t know. I think it’s neat. JAMISON:  Because I’ve been playing with some compiled languages and I always miss NPM. TIM:  Yeah. CHUCK:  Well, then I’ll go next. My first pick is, it’s a blog post by Ernie Miller. It’s ‘Why I Love Being a Programmer in Louisville’ or ‘Why I Won’t Relocate to Work for Your Start-up’. And he talks about why he likes living and working in Louisville as opposed to being in some of the big software centers like New York or DC or San Francisco or any of those. And he kind of outlines a lot of the reasons why when people come after folks that aren’t in those locations, “Why do you live there?” Or, “Why do you work there?” He really outlined a lot of the reasons why I work from home in a location other than one of those places. The other one that I want to pick is an Infographic that the folks over at Crazy Egg came up with. And it’s basically about podcast listening and audio consumption and I thought it was really interesting. So, if you’re a podcast listener or a podcaster or anything like that, then this is probably interesting to you. James, why don’t you give us your picks? JAMES:   Right. So, actually we touched on both my picks. I was going to pick CI.Testling.com which is the thing I’ve been working on lately to make tiny modules to give them good browser tests. And Voxel.js which I’ve also been working on. Then, I guess I could mention CAMP JS where I’ll be next week in Australia. It’s JavaScript’s camping conference-y thing. CHUCK:   Cool, sounds like fun. TIM:  So, you’re like ultra gold status on whatever airline you use now, right? JAMES:   Yeah, I guess. TIM:  Because you’re always everywhere. CHUCK:  Yeah. His Frequent Flyer card is platinum plated. Alright. Tom, what are your picks? TOM:   Alright. I’ve got three for you this time. The first is Discourse which is a form software, open-source form software under GPL license by Jeff Atwood who you probably know from Stack Overflow, Stack Exchange. So, he’s got a new start-up, it’s in Ember.js app. It’s running on top of Rails. Again, open source. And the business model that they’re going for is basically like WordPress. So you can download source code, you can run it, you can install it, you can do whatever you want. If you don’t want to deal with that, you can pay them and host it. So that just launched the other day and I’m really excited about it because it’s the first really big open source Ember app and I think it’s cool to check out. It’s really fast too. CHUCK:   Is it Rails on the backend, because I thought I heard about it from one of their folks too. TOM:  Yeah. It’s Rails on the backend. CHUCK:    Okay. TOM:   The last two things are, as you can imagine arguing about JavaScript on the Internet gets quite exhausting. And what I like to do to relax is cook food primarily for myself because I’m single and lonely. There are two things in particular that have really helped improve my cooking and made it more enjoyable. So the first is the ten-piece glass bowl set from Williams-Sonoma. Now really, any set of glass bowls will do, but it makes me feel like a professional. You feel like you’re on a Food Network show. There’s a technique called Mise-en-place where you basically prepare all of your ingredients up front, ahead of time and then cooking becomes very easy and very pleasant and you look cool too. So, if you have someone over and you’re cooking for them, it looks really impressive. The last thing is there’s a recipe book by America’s Test Kitchen called ‘The Best Simple Recipes’. And every recipe that I’ve made from here is very simple, it takes about half an hour and they have all tasted amazing. They’re like an order of magnitude better than the average recipe you’ll pull of the Internet. So, whenever I’m about to go cook dinner, I just go pull something from there and it always ends up great and it’s easy. JAMISON:  Are you an experienced chef? Or you’re just kind of figuring this out? TOM:   No. Two years ago, I couldn’t even boil water but it’s something I picked up. It’s nice because you’re sitting in front of a computer all day, you don’t really have anything tangible. Making something with your hands is really nice and you can kind of disconnect and then you get a nice meal at the end. It’s really cool. CHUCK:  Awesome. I think you spent some of my money right there on that book. Alright. We’ll go ahead and wrap this up. Thanks for coming guys. It’s been a really, really interesting and awesome episode. TOM:   Yeah, thank you. I had a lot of fun. JAMISON:  Sorry about all the tech problems. I’m glad that substack got on. CHUCK:   Yeah, you got them on. TOM:   Hey, I’ll be back for a rematch whenever you guys want. [Laughter] CHUCK:  Well, I think you both made good points and that’s the thing is, it’s like, “Okay, where do I fall? What kind of projects am I working on? How does this apply?” And there’s real value there. So, thanks again for coming. JAMES:   Thanks guys.

Sign up for the Newsletter

Join our newsletter and get updates in your inbox. We won’t spam you and we respect your privacy.