287

287 RR Hacking the Asset Pipeline with Cameron Dutro


00:40 – Introducing Cameron Dutro

2:15 – What is the Asset Pipeline?

5:35 – Problems and limitations of the Asset Pipeline

8:10 – Cameron’s biggest frustration with the Asset Pipeline

14:45 – Doing it the Rails way, the Angular way, or the React way

20:25 – Keeping your Webpack and Asset Pipeline separate: Working with Javascript and Rails

31:45 – Creating your own preprocessor for putting a file format into the pipeline

  • Sprockets github link

36:15 – Other issues with the Asset Pipeline

40:00 – Causes behind problems with the Asset Pipeline

42:05 – Hygiene of packages

43:40 – Incorporating plugins into the pipeline

45:30 – Resources for learning more

Picks:

Drawing on the Right Side of the Brain by Betty Edwards (Jason)

Philosophize This! Podcast (Jason)

Typora (Brian)

Facer.io (Brian)

Eventual Millionaire Podcast by Jamie Masters (Charles)

Toggl time tracking software (Charles)

Being nice to each other (Charles)

Ruby Together (Cameron)

Lumosity (Cameron)

Seattle Seahawks (Cameron)

This episode is sponsored by

comments powered by Disqus

TRANSCRIPT

Charles:    Hey everybody and welcome to Episode 287 of the Ruby Rogues podcast. This week on our panel we have Jason Swett.

Jason:            Hello.

Charles:    We have a special guest filling in, he’s Brian Hogan.

Brian:        Hi everybody.

Charles:    I’m Charles Max Wood from Devchat.tv. This week we have another special guest and that’s Cameron Dutro.

Cameron: Hello from sunny San Francisco.

Charles:    It’s been awhile since you came on, you talked about Internationalization the last time you’re on. If people want to check that out, we’ll put a link in the show notes. Do you want to just give us another brief introduction? Maybe tell us what you’ve been up to since that other episode.

Cameron: Sure. Still working for Lumos Labs. We make a product called Lumosity, which is online brain training program. You can get it on your iPhone, your iOS device, your Android device are also on the web. I work mostly on the platform teams, I’m still doing that. Still mostly working on internalization and a little bit of performance stuff recently too.

One of the things that spurred I think this call is that I gave a talk at Rails Remote Conf called Hacking the Asset Pipeline. I’ve been working with the asset pipeline a lot since I came on the show last. The asset pipeline had been kind of a pain to work with sometimes and a pleasure to work with other times. I compiled this list of helpful tips and tricks that I wanted to share with the Ruby community. Gave that talk and then was reached out too by Chuck, he’s like, “Hey do you want to come on the show and talk about the asset pipeline?” I said, “Sure”. That’s the path to the show the second time.

Charles:    Yeah. I really enjoyed the talk and I thought it would be interesting to dig into some of the aspects of the asset pipeline. I do want to warn the listeners that this probably isn’t as prepared or cohesive as the talk is, so you probably want to go listen to the talk as well as listen to this episode, because this is going to be more of a conversation about the asset pipeline, what it does, how it works, why we have it, what we like about it, and what we don’t like about.

Cameron: Totally.

Charles:    Do you want to give us a brief introduction as far as what the asset pipeline is,     what it does, and maybe a little bit of how it works.

Cameron: Yeah, yeah, totally. The asset pipeline is the Rails static asset build system. The reason that there’s a lot of contention about it is because the asset pipeline does a lot for you, kind of just like Rails does. In that way, it’s actually a very classic Rails style. If Rails was to make an asset pipeline, it would look exactly like the one we have because of all of the convention over configuration, mantra and ideology that the Rails community has. That’s basically what it is in a nutshell, static assets build system.

                It handles compilation, compiling things from SAAS, or JSX, or ES6 into JavaScript and CSS. It handles in lining so that means like gluing assets together so you only deliver one file to the browser instead of multiple files. It handles like modification too. I remember those different steps, you can throw your assets through that minify them. One of which is called Uglifier for JavaScript which will rename very well, so shorter names were possible, things like that. In CSS, it lets you moves all of the new lines, spaces, and things like that, so it’s modification.

The asset pipeline also does a couple other cool things. It gives you live reloading and development. If you change an asset in development and then refresh the page, that asset will get recompiled by the asset pipeline under the hood and sets your browser so you don’t have to worry about restarting your Rails server or running a build script to compile your assets.

It does a lot of other cool things too, I’ll just touch on these briefly. One of the things it does is it appends a hash digest onto the end of each asset path in production so that if you make an asset change and then deploy, the asset will have a different digest, which means the browser looks like a different file which will bust the browser cache automatically. The hash by the way on the end of that file is a hash of the file’s contents. For example, if you go and then take that change out, your hash will be the same as well before. New clients will also want to redownload these assets again if you roll back for example. There’s a lot of cool functionality that has built in. I think that’s pretty much all that I can think of that I want to touch on for that.

The fact too is there’s an extensibility part too as well, where it supports a bunch of different preprocessors. Like I mentioned, you can process JSX, CoffeeScript, SAAS, and LAAS, and things like that with it. You can also write your own plugins to it, the cold preprocessors that will take basically any kind of file extension, transform the contents and then write that to disk or somewhere. It does have kind of an extensibility element as well. Does that give a good overview?

Charles:    It does. I remember when it first came out I was like, “Oh this is nice. I could just write my JavaScript and it does all the right stuff to it. Builds it and it’s super nice.” And then I got more and more into JavaScript and I was like, “Ah, okay, I don’t want to use CoffeeScript. There are really nice build tools in JavaScript and it just felt like I was more and more limited by it.” Do you hear that a lot and what’s your reaction to that? I’d also like to hear what Brian and Jason’s experiences along these same lines.

Cameron: I not only have heard people talked about the problems they have and the limitations of the asset pipeline, I have experienced them myself too. I think one of the big limitations of the asset pipeline is just speed. At Lumosity our big main Rails app, it’s a pretty big Rails app and without any optimizations our assets probably take over 10 minutes to build and that’s pretty bad. I know that JavaScript solutions to an asset build system will be much faster. That’s one of the big things that I see as a big problem in the asset pipeline experience.

                Also, configuring it can be pretty confusing because the config options are not totally obvious when you read them. That’s unfortunate. Maybe that’s because Rails doesn’t like configuration, Rails likes a convention so maybe we just haven’t in the Rails community have that much experience writing clear config options, I don’t know.

                I talked about the extensibility of asset pipeline but it does have a fairly unstable public interface and that’s like you’re writing your own preprocessor. You have had to convert it on every major version of the asset pipeline. That could be frustrating as well.

    The last couple things, I guess it’s a pretty complex code base as well. Asset pipeline is mainly made up of couple of Gems or the biggest one that was called Sprockets. Sprockets isn’t really complex code based, it’s kind of hard to figure out what’s going on in there. There’s been some clean-up efforts recently but not necessarily as much as I would like to see.

The last thing I have an issue with the asset pipeline is it’s hard to debug. You don’t really know where the stick of binding that pry to find out why your assets are building, or why your assets are building incorrectly, or whatever.

Brian:        I’ve always got the impression from working with the asset pipeline. For some reason, it doesn’t seem as polished as the rest of Rails, it always seems that that’s the area where I run into problems. That’s the area where other developers run into problems. I was wondering because you’d mentioned that you’ve run to some frustrations. I was interested to know what is the biggest frustration that you have repeatedly with the asset pipeline and how did you resolve that.

Cameron: Okay, that’s a great question. It’s kind of two-fold for me but I would say consistently the biggest problem is just speed. It’s just an unbelievably slow compilation system.

                As I’ve mentioned, speed I think was the biggest problem that I’ve had. That’s not saying there haven’t been other problems but speed is by far the big one. Whenever you’re trying to debug something on the asset pipeline trying to figure out why your asset won’t pile or whatever. Usually what you have to do is to a bunch of test compiling, test precompiling I should say. If that process takes seven, eight, nine, ten minutes then you can be stacked in this really slow feedback loop. That’s a lot of the pain that I have suffered through and trying to do certain thing to the asset pipeline.

                In the talk, I do talk a little bit about how I sped things up in the asset pipeline, or how our team sped things up, it wasn’t just me. Let me actually refine my slides on that because I can list up the things that I talk about here. One of the big things that the asset pipeline or that Sprockets specifically suffers from is that Ruby is not a great environment in which to execute JavaScript because obviously Ruby and JavaScript are totally different runtimes, totally different languages. What the asset pipeline does to mitigate that problem in order to run things like Uglifier and other JavaScript tools, it uses an embedded JavaScript runtime called the Ruby Razor. In J Ruby that’s called the Ruby Rhino which is a wrapper around Java library called the Rhino which is a JavaScript runtime in Java.

You have all these kinds of levels of indirection. This is one of the big reasons why a lot of people I think are moving to build systems like Grant, or Gulp, or webpack, or something in order to do their static compilation because not only is it faster, it’s also like Native, it’s not running through any kind of interpreted levels of interaction there. One thing you can do though and something that we used to, we got about a 7% speed increase here was we use Node instead of the Ruby Razor. If you install Node on your system and just take the Ruby Rails rather to Gem file, this other Gem called exactJS which is between kind of seamless place between JavaScript runtimes will automatically Node is that you got Node installed and use that instead. That results in about 7% speed increase. At least it did for us.

                Another thing we did was we disabled gzipping. The asset pipeline is capable of gzipping all your assets which is generally a good idea except that doing that in Ruby is not as efficient as just letting just your CDN do that. If you don’t have a CDN then you would definitely want to keep this enabled but we do. When we upload our assets to the CDN, whenever those assets are requested, CDN will automatically serve them gzipped. We didn’t need to do that in Ruby Lan. I think that’s saved us about 2% time. We’re up to about 9% total. They’re not huge speed increases but definitely not nothing. A pretty significant when you think about the overall time.

                Another thing we tried to do was use this thing called Lib-SAAS, we have a lot of SAAS in our app. SAAS being the CSS preprocessor language, that’s a bad word for it.  Who can describe this, it’s a better CSS basically.

Charles:    Yeah. It’s a more powerful and forgiving CSS but it has to be transpiled similar to TypeScript, or CoffeeScript, or something like that into JavaScript or in this case CSS that your browser can actually run.

Cameron: Yeah, exactly.

Charles:    Yeah. Lib-SAAS is the C implementation of what used to be running Ruby and its way wicked fast.

Cameron: Yes, exactly. Lib-SAAS is like the C implementation, exactly. We tried to put that in our Gem file, that didn’t work because of Compass and Bourbon which have Ruby code embedded in. They have to execute Ruby code whenever an asset gets precompiled, which of course you can’t do in a C context or maybe you can but it’s not that supported. We had to abandon that. That probably would have sped things up a lot for us though.

I think by far the most speed increase the best performance optimization we wanted to make was compiling assets in parallel. If you can somehow harness all those different cores of your laptop or your build machine, then that can really speed things up. This is going to be temporary by the fact that didn’t actually work. But we saw like a 47% speed increase compiling assets in parallel. I say that it didn’t work because it ended up skipping a couple of assets and I ended up kind of abandoning that approach because I wasn’t spending too much time on it but it didn’t really make sense to keep going there. If somebody out there is willing to pair with me, I’m trying to figure that out, that would be awesome. I also want I mention, I think Sprockets 4 which again is the Gem that underlies the asset pipeline, I think Sprockets 4 actually does this better. I think it does compile assets in parallel. Maybe the answer for us is just upgrade to Sprockets 4. Those are kind of the big things

I guess there’s one more thing that we implemented which is asset fingerprinting. What that does is basically skips the entire precompile process if your assets haven’t changed at all. The way that it does that is it looks to all of your assets, so Sprockets expose an asset load path, you can explore all those assets and then it reads all those files and throws them into a hashing algorithm, and then writes out the resulting hash digest to an asset fingerprint text file. The next time you run asset precompile, the same hashing is done, and then we compare the current fingerprint with the previous fingerprint. If they match, you just don’t precompile.

Those are the five optimizations we tried and I think for the most part they work pretty well. There were a couple of dimensions that worked. I think fingerprinting is by far our biggest speed increase because of course nothing actually happens if your assets haven’t changed then that like is maybe that’s probably the 75% of all our deploys. Does I give you guys a kind of idea of my biggest problem with it and how that was solved?

Brian:        Yeah, absolutely. It was nice to hear about the gzipping [00:14:37] those kinds of things to the CDN, other practical things.

Jason:        I have a question. I work a certain amount with Angular and there’s kind of two ways to use Angular with Rails. One ways to put it in the asset pipeline and another way is to not to use the asset pipeline but use a build tool instead and have a client server architecture. In my mind, it’s kind of like an either or type of thing. I do it one way or I do it the other way. This might be a dumb question but are those two ideas mutually exclusive? Do you think it could ever make sense to use a build tool like webpack and the asset pipeline or just it really just makes sense to use one of the other?

Cameron: Yeah, that’s a good question. Personally, I think it make sense to use one or the other just because I think there’s a lot of room for human error there. You could end up including assets twice because webpack could reference an asset, the asset pipeline could reference an asset, it could both end up compiling that asset. If you’re gluing all your assets together somehow into like one big JS file, one big CSS file, it ended up including asset twice.

I don’t know about webpacks, I haven’t used it really at all but I do know that the asset pipeline will have [00:15:58] those includes for you, they call them preprocessor directives I guess in the manifest file or in the JavaScript and the CSS files that are included in the asset pipeline’s precompiled list. I think it makes more sense to use one of the other, I don’t know, does that kind of ring through, what do you guys think?

Jason:        I think still because that’s the way I’ve been doing it the whole time. I either do it one way or do it the other way. I just wonder to maybe there were something that I didn’t know.

Brian:        My experience has always been that when playing in Rails Lan because I’ve been working with Rails since 2005. Every time I’ve deviated from the way Rails wants me to do something, it’s always comeback to bite me because unfortunately I end up maintaining my old code bases once in awhile. I’ve forced myself to just do the work with what I’m doing with React, right in the asset pipeline when I was doing stuff with Angular work right into the asset pipeline. I’ve actually never tried the other approach but I know that there are colleagues of mine that have so I’m always interested to hear about the experiences with that. I’ve always gone with, “The asset pipeline is here, that’s the way I should do things. I’ve learned my lesson in the past. I’m just going to use the asset pipeline and make it work.”

Jason:        I think there’s definitely some ways to swimming with the current and doing things the way the rest of the world does it. Because if you do stuff the same way other people do it then you’re more easily find like answers to your questions because other people are having the same kinds of challenges that you are and likely somebody has solved your problem before. I definitely agree. That’s a good idea.

Charles:    Yeah. I just have to say that at the same time I’ve been playing with React a lot lately and I’ve done quite a bit of Angular, I have a podcast about Angular. I have to say that for me it’s not really, “Oh, well I’m in Rails. I’m going to do it the Rails way.” Especially with the Angular CLI and some of the React CLIs that are out there, they do a lot of the setup for you and they go the other way, they actually use webpack. I wind up being in the position where it’s like, “Okay, do I do it the Rails or the Angular way? Do I do it Rails way or the React way?”

Jason:        In that situation you’re arguably no longer even in Rails. The way I do it, I have a totally separate folder that is completely unaware of Rails. You don’t have to take that in the consideration anymore in my mind because you’re not even really in Rails anymore.

Charles:    Yeah. I generally go that way. I actually at least for the JavaScript and to some degree for the CSS depending on which features I need in the precompiled because webpack has some nice features with modularizing your CSS and things like that, and the way that it sets things up. Sometimes I want the JavaScript tool because it’s easier to hammer that out. With the JavaScript, I will admit that I almost always opt for the webpack, automatic setup thing, whatever it is that comes with the JavaScript framework that I’m using that at least for my JavaScript.

Camero:    What sort of things do you think you lose by doing that? What sort of advantages do you lose by keeping them separate?

Charles:    The big thing that I see and it’s becoming less and less of an issue because most of the frameworks actually incorporate some form of DOM management is the jQuery Rails integration, that’s kind of nice. Sometimes if you’re building a simple or simple ish app and you really only need the jQuery stuff but you want some of the nice stuff that comes with React or Angular for manipulating the frontend, then that’s where I wind up making the trade of is that, “Oh, I could do this really nicely with the remote through and some slick little call back that Rails has built in that works with jQuery.” Then I have to pull the Angular stuff in another way.

                The other trade-off that I’ve seen and I’ve only run into this a couple of times is that I want the latest JavaScript in the Gem that pulls it in and vendors the JavaScript isn’t always up to date or doesn’t have the capabilities or other libraries that I want. I wind up doing some kind of hybrid Browserify, NPM mash up with the asset pipeline.

Jason:        There is a question I want to ask you Cameron is the management of JavaScript libraries. There’s something that I really hate which I called the JavaScript in a Gem pattern, where there’s always Gems that exist solely for the purpose of wrapping some JavaScript library. I don’t think that’s a very good way to manage JavaScript libraries. One alternative is to use Browserify, I use this Gem called Browserify Rails, I believe it is. That allows you to use NPM to manage your JavaScript and stuff, or you can use power but it seems like that’s kind of falling out of favor more. Do you have any thoughts on that part of it Cameron?

Cameron: I do, I do, yes. I completely understand where you’re coming from. The pain of managing JavaScript dependencies especially since like if you use NPM then you get a lot of things for free, you get all of the dependency resolution, you get everything comes with that, that’s really nice. I don’t think that can really be overstated, that’s pretty cool. Just like Bundler and Ruby Gems, you drop a Gem to your Gem file, you bundle and it brings all the dependencies with it, just kind of automatic works and Bundler sets up your load path for you, it’s awesome. I think that we would probably be pretty well served if we use NPM to manage our JavaScript in Rails.

There’s a couple of approaches I have seen that do to that pretty well. Personally, I’ve never encountered, I can say never but I frequently don’t encounter a problem where I do have lots and lots of JavaScript dependencies. Most of the time my Rails app handles most of the, usually I’m sprinkling JavaScript on top of a Rails app. For example, putting React in there and rendering some templates. React doesn’t really have any dependencies, or jQuery, or like any of the charting libraries. Even though I just mentioned how dependency resolution is great, I think that’s part where we’re eventually going. Ruby and Rails, let’s just say JavaScript and Rails developer will probably be better served using NPM going forward. Like what I’ve done in the past, is use this cool project called Rails assets. You mentioned how JavaScript, lots of these Gems exist and all they really start do is wrap a bunch of JavaScript and then provide it to the asset pipeline via Rails engine or a Rail tire something. Having to maintain all that especially if you’re a maintainer of the Gem like that, it kind of sucks because then you have to remember to go up at the JavaScript whenever that changes. That means all the developers that depend on that Gem, if a new version comes out the Gem is updated there, it’s going to be frustrating for them. Definitely a problem. I think that Rails assets though does a great job of automating that. If go to rails-assets.org, I’m pretty sure it’s still up.

Jason:        Yeah.  I’m looking at it right now actually.

Cameron: Basically what it does is it provides an automated Gem builds for JavaScript libraries that NPM serves. The first time you ever request a Gem or request a version of a JavaScript library in your Gem file, it will automatically build that Gem and give it to you as a Rails asset Gem, which is pretty cool. You can actually end up managing all your JavaScript libraries via Bundler. Whether or not that’s a good idea, I don’t know but it works pretty well for me in the past. I’m a fan of it. It does let you bypass the whole NPM problem.

Jason:           Yeah. I could see that.

Charles:    Yeah. People are up on stuff than it really does just work or if there’s not that critical thing that fixes a bug or adds a feature then again, you’re fine. I don’t experience this every time I pull in a vendor to Gem version of a JavaScript library but it does happen on occasion.

Cameron: Right, for sure.

Brian:        Yeah. The challenges like letting Rails be helpful with assets and also maintaining like a same separation of concerns because it’s easy to go too far down the path of mixing the Ruby stuff and the JavaScript stuff. Things can get a little weird, like I said, when you have all these JavaScript in a Gems that feels a little bit funny because it feels like the Ruby stuff is too aware of the JavaScript stuff that it shouldn’t be. It doesn’t sound that Rails assets project suffers from quite the same problems.

Cameron: Like you said, if your Rails app is too aware of your JavaScript and it feels dirty like your mixing concerns that shouldn’t be mixed. That’s kind of what the asset pipeline was designed to do. It was designs to bring all of these disparate technologies like CSS, and JavaScript. Even to some extent like fonts and things, it was fun to take all that stuff and make it easy to integrate into a Rails app.

There’s definitely a lot of value there. It’s one of the things like whenever I go to another framework, like Django or even Node, one of the things I really miss is just that automatic working set up asset build system that just already there and working. I do hear the problem where entering a JavaScript can be a pain or like having a Gem can be a real interesting strange cross over there. I also think that it can be really handy when all you want to do is spend up a quick app or you want to interrupt some JavaScript and you don’t want to have to figure how to download it and include it, and then put it in asset pipeline yourself. You are just able to depend on a Gem from Rails assets like Rails asset jQuery or Rails assets React, then that’s just a lot simpler, and a lot more straightforward, and a lot more Railsy, I hesitated to even use that word but a little bit more Railsy than doing it manually.

Jason:        Yeah. I would agree. If I were doing a new project today, I have this potential project coming up that’s like a warehouse management project. I don’t see that as having a lot of fancy UI needs. If I were to do that project, I would probably just do a regular Rails project without front-end framework like React or Rails or anything like that and just use the asset pipeline. Because I think for the use case where it works, which is like in my opinion most applications, it works great. It’s just for those applications that do a lot of JavaScript or things get a little weird but for most applications, they don’t need a lot of JavaScript. I think it’s fine.

Cameron: It’s always interesting when that topic comes up, we talk about applications that need a lot of JavaScript and applications that the sort of modern and real time types of applications where you really are going to have sort of DOM API in the backend and you don’t even need the whole stack that Rails provides, a very heavier front-end application. They always hear developers, “Guess, that’s where the web is going.” It’s always interesting to hear from other developers to say, “No. To be honest, 80% of work I do is boring CRUD applications.” That’s one of the interesting thing about the asset pipeline is I almost wonder if the asset pipeline was in fact meant to solve that problem. To solve that problem, we need to add some JavaScript to our CRUD application. JavaScript is not the star of the show. Our forms, CRUD application is the star of the show. I always wondered that because it doesn’t seem like it’s keeping pace with what’s happening with React, Angular, the frontend, and the JavaScript Lan.

Brian:        Think about it from this perspective, the same project I talk about. If I do this, I’m going to want to spend something up in two weeks and deliver a minimal product for the client. I don’t want to spend a lot of time on it and I don’t want there to be a lot of technical risk involved in it. I would definitely just use the asset pipeline in that case. And then there’s also the consideration of like if you know what you’re doing then it’s not a problem really to integrate JavaScript frameworks and stuff like that but you’re not always going to be working on a team of like a bunch of senior developers. Often there’s going to be a lot of junior developers who can very easily and I’ve seen this a lot, very easily shoot themselves in the foot by using a bunch of these technologies together. And then you just end up with a big configuration nightmare. If you’re using the asset pipeline, then there’s one way to do it and you’re not exposing yourself to all that technical risk that you may have otherwise.

Jason:        Yeah. You’re describing exactly the way Rails works. You need something to [00:30:39] click when it brings up Rails application.

Charles:    So true. One thing that I’m hearing here though is that if you’re bringing in a library that does a lot on your frontend, you probably want to look at maybe some of the JavaScript tools for builds. If you’re sprinkling your JavaScript in, I like the way you put that Cameron, then maybe you’re looking at the asset pipeline and you can make the decisions based on those trade-offs going from there.

One thing that I’ve seen that I ran into a while back was I decided, oh, I’m going to pull Angular in through the asset pipeline. And then all of the Angular 2.0 examples if you go and look, at least the ones that are updated first are all in TypeScript so I was like, “Oh, I want to use TypeScript for my stuff,” so I pulled in Rails TypeScript, or TypeScript Rails Gem, I don’t remember what it was and it totally died. I tried to compile stuff and it just wouldn’t play nice.

I’m wondering, let’s say that instead of TypeScript which is probably fix now. Let’s say that I’m writing my own transpiled JavaScript language called ChuckScript and I’m like, “Dude, ChuckScript is the stuff and I want to use it with Rails. I’d like asset pipeline to work with it because CoffeeScript uses CoffeScript anymore.” How do I build something like that so that it’ll put that in so that the frameworks that are out there that really want to use my particular flavor of JavaScript transpilation can do it and have all of the sprinkling in that I want at the same time?

Cameron: Yeah. That’s a great question. That gets to part of the talk that I also gave which was creating your own preprocessor for whatever file format you happen to want to put in the asset pipeline. First, I want to say though, I would totally use ChuckScript, that sounds awesome.

Charles:    It does everything you want, I promise.

Cameron: Yeah, cool. I think that this is probably good candidate for that for adding this preprocessor. The description about how to do that, it’s actually not mine, it was written by Shenim. I can probably send you guys a link to this. It’s on GitHub, it’s a markdown file and he just describes how you write a preprocessor so that it works with all major versions of Sprockets. That’s version 2, 3, and 4 for the most part. 4 being the most cutting agent, that’s in Rails 5. You can use it with Rails 4 as well but not without significant work there.

                Basically it’s a pretty straightforward interface, this is for Sprockets 3 and 4, you create an object that responds to call. I think it receives a context variable or something. You pull in information out of context and including those source of the asset that you want to compile, and then you would take the contents that asset, throw it through the ChuckScript precompiler or transpiler, and then hand that back to the asset pipeline by returning it from that call method, and that gets written to disk and do your business.

Charles:    Very cool. I would have to have Node installed if that’s a prerequisite for the transpiler for ChuckScript. And then have the ChuckScript transpiler CLI tool installed as well, and then I just set up my asset pipeline plugin to essentially call out to the ChuckScript command line tell which files to pick up and then it sends it back to the plugin, and then it goes on to the next step in the asset pipeline.

Cameron: Yes, yes, exactly, that’s right. What you probably do, and this is again putting JavaScript in a Gem but you probably have the ChuckScript-Rails Gem which would handle all of that interoperability for you so that it would take the file, transpile it using the CLI, you would use exact JS probably or something like that to transpile. That would be a whole separate Gem and all of the person would have to do if they want to use your Script is to just drop that in a Gem file and then create some ChuckScript.csh or something, I don’t know what the file extension would be. Some files insert in their app assets directory.

Charles:    Right. What require me to install Node and then do an NPM-g installed ChuckScript.

Cameron: Exactly, yeah, yeah.

Charles:    All that transpolation would just be in an executable JavaScript that does its job.

Cameron: Exactly, exactly, yeah. That’s kind of the magic of the asset pipeline. You don’t just drop crap into your Gem file. This is also maybe part of the reasons it’s bad but you can drop crap into your Gem file and then for the most part it just works.

Charles:    Nice. By the way the file extension for ChuckScript is .awesome.

Cameron: Oh, right. Sorry, I should realize.

Brian:        Sorry, I’m already using that for awesome script so can’t’ have that.

Jason:        I thought it was .cmw, I think that would make sense.

Charles:    Oh, there we go.

Brain:        Yeah, there we go.

Charles:    I sign everybody else’s work, that’s why it’s cmw.

Jason:        Awesome. I’m curious, I describe some of the issues that I have deal with the asset pipeline but I’m really interested to hear from you guys what other issues you’ve ran into, maybe there are performance related. I know we talked about performance, we talked about any of the issues that you ran into when you want to add new steps to your app and things like that. Are there any of those specific problems that you guys to run into using the asset pipeline?

Charles:    I was just going to say the only other one that I’ve ran into is when I use some of these plugins it doesn’t always do hot reloading, which if you use CoffeeScript or something like that. I updated, I refresh my page on localhost 3,000 and I get updated JavaScript and I don’t always get that development note.

Jason:        Interesting.

Charles:    Anyway, what were you going to say Brian?

Brian:        The biggest issue that I always run into is when working with junior developers or people new to Rails. I spent a lot of time teaching people how to use Rails. Everything is going along just fine. These are people who typically have a background in html and CSS. They want to move into the backend. The issues they ran into by the way are, “I want to use bootstrap, how do I use bootstrap?” “I know I will find the bootstrap Gem.” “Wait, which bootstrap Gem do I use?” “I have to install this and where do I put these lines of configuration.”

A lot of times what I’ve seen is the people who are really excited about putting things into plugins, Gem plugins for the asset pipeline, aren’t necessarily keeping pace with things that are changing. I see this a lot with a lot of the third party things that coming out. That’s what Chuck was saying. We’ve got an error that comes up, we’ve got bugs to get fix, or whatever. A lot of these just come into the asset pipeline itself is fine but all of the JavaScript add ins and things like that seem to be a bit of an issue, either behind or they’re buggy, or there some piece of documentation missing, or it’s a little bit vague and what were supposed to put a specific configuration setting. I’m getting back again to what we’re talking about earlier about convention over configuration. I think that’s the number one issue that in run into when working with junior developers. The number one fix I see a lot of people doing to get to run that, they simply copy links in from a CDN instead. I think that’s interesting.

Cameron: Yeah. That’s interesting. Does that work for the most part?

Brian:        Yeah. It works because you just them in the layout file. “I got bootstrap, I got bootstrap working, it’s done.” They just kind of bypass the asset pipeline all together. “Did you use the bootstrap Gem?” “Nah, I couldn’t get it to work, what did you do?” Out there it is and there, all right. A little I guess but it speaks to a problem because that’s the thing is we get so close to this technology. The more experience we get with it, we see those problems and we immediately fix them, we can immediately address them. The junior developers are the ones that are coming up. They haven’t seen those kinds of weird issues before. I’d like to see everybody who really knows this stuff to do their part and sent poll request in to make documentation on these things better, and give more wonderful presentations like our guest has given, and to just really educate people and help people really understand how to use the asset pipeline.

Jason:        Do you think most of the problems that arise with people who don’t know the asset pipeline as well as just that they don’t know it as well, or because it’s too complicated, or is there some other root cause to that?

Brian:        I can only guess but it’s a lot of like what happens in other parts of Rails. If you just stick with stuck Rails stuff, things are pretty smooth but as soon as you start bringing in third party Gems, things start to get a little bit more interesting. The more of those you add, I’ll think about what today’s, let’s say today’s users want in their applications. Then think about what developers have to bring in to make those things happen. We are a long way, way from a simple web application. Now we need lots of additional libraries to make things look good, make things work responsibly, and get people that nice real time feeling they’re having. The more these dependencies we had at the project, the more trouble and compatibility issues it may run into especially for wrapping JavaScript inside a Ruby. That’s my opinion on that. That’s not too far off of an opinion I think because we all know that the more dependents we had to a project, the more maintenance had expert going to potentially introduce.

Cameron: Yeah. Okay, that makes total sense to me. I think that’s true, Gems as well as JavaScript, like you add more complexity to your app and that’s just compounds the issues of maintainability with that app. We definitely have seen that at Lumosity. We have I think something like 460 Gems in a Gem file which is just like to me is astronomically big. We also have a lot of JavaScript in there. It’s kind of varying levels of professionalism that it’s written in because we’ve had a number of junior developers as well as senior working on the app for something like six or seven years. You definitely build up this mountain of technical debt.

                This is totally off the topic of the asset pipeline specifically but one thing I would like to say is that we as Gem developers, we as JavaScript library developers should always be thinking I thinking about the hygiene of our packages. What I meant by hygiene is making sure that we version them with semantic versioning so that breaking changes are obvious. Also, making sure that we don’t do things in Ruby like patch core classes, like string, or hash, or array, things like that. Just a quick shoutout to that. I think a lot of cases help with compatibility. I don’t know how many times I’ve tried to track on a really weird bug to only to find out that some Gem has patched string to assets or something.

Jason:        Exactly. The hygiene you’re talking about is does your code play nicely with others.

Charles:    Yup.

Cameron: Right. Yes. Sandi Metz says, one of her big topics I think and I’m going to be in trouble if I’m misquoting here but I think she has really stressed that when you write code, and this is true, any language where you really optimizing force not necessarily speed, well at least not at first, not necessarily speed, I wouldn’t even say readability so much, it’s just making it easy to change because you know that it’s just going to be a fact, it’s just a reality that you’re going have to put change that code in some point in the future. By writing that code with that in mind oftentimes you can produce I think better code that won’t give somebody headache down the road.

Brian:        Getting back to the issue of these plugins. Let’s say you have a JavaScript library, let’s say you have this brand new JavaScript library, there is no Gem for it yet but you’ve really want to use it on your project. What do you do to incorporate that in the asset pipeline in the most non-destructive and easiest way?

Cameron: Okay. That’s a really good question. The thing that I would probably do is first ask, is this library something that’s on NPM already? If it is, then I would just use Rails assets. Rails assets-package name and pull that in as a Gem. I think that’s probably more sustainable. Because then every time that a new is published to NPM, you can request that new version and Rails assets will build that Gem for you and deliver it to you. If it’s not an option, if it’s not published to the NPM, the easiest way is probably just to vendor it. I don’t like that strategy so much because you don’t really make and there’s no updates that you get automatically or easily. I think that’s your last resort I think is to vendor it.

Brian:        Yeah. You’re saying that the Rails assets approach is preferable to vendoring whenever possible?

Cameron: Yeah. I think so because you just get a lot benefits from that. Most of the benefits are related to versioning.

Brian:        Fantastic.

Charles:    I’m wondering Cameron, let’s say that somebody decides, “Hey, I want to dig into the asset pipeline and really get it.” What resources did you use to figure this stuff out? Was it trial and error, or did you read the code, or have you read blog post or other books?

Cameron: That’s a good question. I have not really read a blog or written blog post. Most of my experience to the asset pipeline has been just trial and error. I also did read the Rails guides. When it first came out and like Rails 3.1, there are also like a lot of release notes and that time I was really excited about Rails 3 so I did a lot of reading about it, all these various blog post and things. I’m not going to be able to tell you which ones of those work because that was like probably four years ago. That as a wild go but the Rails guides to the asset pipeline is pretty good. That explains what it is, how to use it, not necessarily how to write your own preprocessor but it does tell you what the config options mean and things like that. Definitely recommend the Rails guide, I think its railsguides.org or something, I don’t remember, guides.rubyonrails.org.

Charles:    That sounds right.

Cameron: That’s a great resource. Just in general, those are really good resources for learning Rails. It’s written in I think a pretty conversational style. Doesn’t have all the information that you might need in there but it does have a lot. The other way that I’ve learned a lot about Sprockets specially is by actually opening up the Gems and looking at their source code. I have to admit that it’s not for the pain of heart, those Gems are really complicated. They’ve been cleaned up a lot by Shenim or Scheiman, these are great job of cleaning those up.

                Recent version of Sprockets like 3.7 and forward or a lot cleaner to look at, easier to understand. You’re still definitely going to be waiting to through lots of new concepts when you first open those Gems. That is the ultimate way of learning how something works by actually reading a source code. Those are the major ways that I’ve played with it and learned about it.

Charles:    All right. That sounds really cool. That kind of sounds like learned it the hard way.

Cameron: Yeah, yeah. I think so. That means that a lot of errors that I made, the issues that I ran into are ones that I won’t make again.

Brian:        Exactly. I just wanted to say that I think what you did is super important. Cracking the source code for the things that you’re using is something that not every developer does but it’s certainly is something. I encourage every developer to do it every now and then, just look at the source code for the things that you sue and you rely on. It’s there for you and you can learn a lot from it.

Cameron: Absolutely, so true.

Charles:    All right. Let’s go ahead and hit some picks. Jason, do you want to start us of with picks?

Jason:        Sure, I got some picks. My picks by the way pretty much never have anything to do with computers. Just be prepared I guess, these are totally random things and I pretty much always will be. My first pick is this book called Drawing on the Right Side of the Brain. That was a really book because it teaches you how to draw even if you think you don’t have any artistic talent. I went through and did their exercises. They kind of spoon-feed you some exercises, they have you draw this grid on a piece of plexiglass and stuff like that. I was just blown away by what I was able to produce after I went through those exercises. If you’re into that kind of stuff at all, check out that book, Drawing on the Right Side of the Brain.

                The other thing is this podcast that I really enjoyed although I haven’t listen to it much lately called Philosophize This! Philosophy is kind of like a notoriously inaccessible topic. This podcast, the host is really awesome, he really spells things out in a very accessible way, makes things really easy to understand, and it’s just super easy to listen to, so I really enjoyed that podcast, it’s called Philosophize This! That’s my picks.

Charles:    All right. Brian, what are your picks?

Brian:        All right, what are my picks? I do a lot of writing, I do a lot of tutorials and things like that. I got to say that one of the things that is saving me an incredible amount of time is this brand new markdown editor called Typora and you can find it at typora.io. It’s a little bit interesting take on the what you see is what you get because you just typewrite inside of it and it converts your markdown right to html markup when you focus away from that section of the document. The biggest advantage that it has is that it supports the mermaid diagramming language. If you need to include flowcharts, or state diagrams, or [00:50:36] in your documentation, you can do them in mermaid code as if you were doing any other kind of code fence and type work and run through the actual document. If you run to the actual diagram right in the document, when you export it to a PDF, or an EPUB, or something, there is your diagram right inside of the document. I think it’s just fantastic, it’s saving me so much time. I don’t have to reason for a vector graph tool for simple diagrams and things like that.

                The other thing is in the month of November, GitHub is doing this game jam. Build the game and submit it to GitHub for judging. I’m participating in that using a web framework called facer.io. I love Facer for making 2D games and I recommend anybody who was never written a game or never dabble in making game, check out Facer because it’s a lot of fun, does a lot of things for you but that’s crazy fun to build some kind of cool games with it.

Charles:    Very nice. I’m going to jump in here with a couple of picks. The first one is I just hired a new business coach and she has a podcast. I’m going to talk about her for a minute. Her name is Jamie Masters, she used to be Jamie Tardy and her podcast is the Eventual Millionaire. I’m super excited to get things going there. Anyway, she only interviews millionaires and talks to them about basically how they made their money and gets a lot of great advice for people who are looking to become millionaires. If you want to check that out, go check it out.

She also turn me on to a tool, actually she gave me homework. That’s what you get from coaches. The tool is called toggl.com. It’s a time tracking software. Back in my days of being contractor full time, I hate tracking time, I seriously hate tracking time. It’s been really interesting. I’ve been only doing it this week, so yesterday and today. Today is November 8. Anyway, it’s just been interesting to see, “Oh, I tend to get side track in the morning. I didn’t realize that until I started tracking my time.” Or “Oh, I don’t really count this time when I start thinking about the time that I spend doing stuff. For example the time I spent planning and fixing up my calendar and doing kind of the email triage I do in the morning for about 20 minutes.” Because that’s about how long it takes me to go through the emails and go, “Okay, archive, archive, archive, archive, quick response, archive, quick response, quick response, archive, oh, that’s an emergency, I got to handle that.” Anyway, it’s been kind of interesting to say, “Oh, okay, I actually spend this amount time doing these kinds of things throughout the week.” I’m really looking forward to what kinds of breakthroughs I’m going to have just from the standpoint of seeing how my time is spent.

I used another app called DeskTime before and it kind of automatically tracks stuff. But it only tracked it based on the applications and website you’re using at the time. I found playing Clash of Clans on my phone or something like that then it won’t pick it up. With this, if I’m being honest then I’ll put Clash of Clans in and go, “Oh, I spent an hour yesterday attacking other clans or whatever.” It’s been really, really interesting just to see where the time is going, so I’m also going to pick that.

Finally, today is November 8, which here in the United States means that people are going to be going out on voting today. By the time this goes out we already know who the next president is, who are congressmen and women are, and who the senators are, and things like that. Things have gotten really ugly during this election. I’ve actually posted some things to my Facebook timeline that other people took offense to. I get a, “Shame on you,” and all that stuff. Ultimately, I just wanted I guess pick, be nice to each other. Regardless of who wins and whose president and all the stuff, I don’t understand why you would think this. I’m trying to picture in my head what your point is instead of, “Oh there’s one little thing about this thing that really ticks me off.” I think we can have some interesting conversations and come to the place where we understand each other. I think this is important not only in the political realm but on our development teams, and with our neighbors, and other people who we interact with at various levels. Really, I guess I’m just calling for people at this point, I know this going to come a couple of weeks after the election, to just come together and talk to each other and try to understand each other.

Cameron: What about like tabs versus bases?

Charles:    That’s a step too far, sorry.

Brain:        No, don’t, no. Next to be in [00:55:41], we don’t need to go there. Oh man, oh man, holy war.

Charles:    Yup. Some things like Tabs are just wrong. Just kidding. All right. Cameron, what are your picks?

Cameron: Okay. First of all, Chuck, I really appreciate that you’re saying no matter what your political beliefs, make sure you’re nice to each other. Everybody has different point of view, it’s super important to recognize that and not just shove your own discoursed on people’s throats or just be rude in general. Totally agree with that. This has been a super-heated election cycle and I think a lot of us are just ready for it to be over. Anyway, we’ll see what happens. I guess a lot of people are a little nervous, I’m kind of over the nervous now, I think I’m ready just to move forward.

Charles:    I’m not nervous, we’re screwed either way.

Cameron: Right. My picks today, I’ve got a couple of them. The first one I want to pick is Ruby Together, it’s that rubytogether.org. It’s this organization that is committed to supporting Ruby infrastructure that we rely all the time like Bundler, RubyGems, other shared tools. They’re doing great work like they’re trying to raise money in order to pay developers to work on this critical infrastructure. Rails generally has a lot of support from the community but Bundler, RubyGems  other short tools, they don’t necessarily have all that support because the people that maintain them have other jobs to do, they’re not paid to work on these things. Ruby Together is a way to help pay for them to do that.

                You can join as an individual developer or you can ask your company to join. There’s different giving levels that you can do. I would totally recommend, I became an individual contributor couple of weeks ago. I just feel really good about doing that because I know I use these tools every day and so does the company and we rely on them, so why not pay for them. That’s Ruby Together.

                I also have to pick Lumosity. Lumosity is the phenomenal way to entertain yourself and to keep your brain trained and elastic, get ready for the next challenge. We have a pretty dedicated science team here that trust to validate our games and prove that they are good for keeping your brain sharp. We publish several studies. We can’t make the claim necessarily that it will make you smarter, anything like that but I can definitely say that the games are fun and that I really appreciate working here. I definitely pick Lumosity games.

                I think my last pick is going to be the Seattle Seahawks. I’m from Seattle and I’m a huge Seahawks’ fan. Go Seahawks. Hopefully they can be the patriots next week.

Charles:    I was going to make some noises about the Seahawks, “Oh man, but if they’re playing, they’re patriots.”

Cameron: I know.

Charles:    If people want to follow up with you, follow you on Twitter, see what you’re doing, anything like that, where do they go Cameron?

Cameron: You can follow me on Twitter @camertron. I’m pretty much camertron everywhere on the internet, that includes GitHub. My GitHub page is much open source projects on it. Those are the two big ones. That’s what you can do.

Charles:    All right. We’ll go ahead and wrap this one up. Thanks again for coming. We’ll put a link to your talk from Rails Remote Conf in the show notes and we’ll catch you all next week.

Cameron: Alright, thanks guys.

Jason:        Bye guys.

x