222 RR Rails 5 with Sean Griffin

CHUCK: Alright, let's go ahead and get started.[This episode is sponsored by Hired.com. Every week on Hired they run an auction where over a thousand tech companies in San Francisco, New York, and L.A. bid on Ruby developers, providing them with salary and equity upfront. The average Ruby developer gets an average of 5 to 15 introductory offers and an average salary offer of $130,000 a year. Users can either accept an offer and go right into interviewing with the company or deny them without any continuing obligations. It’s totally free for users. And when you’re hired, they give you a $2,000 signing bonus as a thank you for using them. But if you use the Ruby Rogues link, you’ll get a $4,000 bonus instead. Finally, if you’re not looking for a job and know someone who is, you can refer them to Hired and get a $1,337 bonus if they accept the job. Go sign up at Hired.com/RubyRogues.]**[This episode is sponsored by Codeship.com. Codeship is a hosted continuous delivery service focusing on speed, security and customizability. You can set up continuous integration in a matter of seconds and automatically deploy when your tests have passed. Codeship supports your GitHub and Bitbucket projects. You can get started with Codeship’s free plan today. Should you decide to go with a premium plan, you can save 20% off any plan for the next three months by using the code RubyRogues.]**[Snap is a hosted CI and continuous delivery that is simple and intuitive. Snap’s deployment pipelines deliver fast feedback and can push healthy builds to multiple environments automatically or on demand. Snap integrates deeply with GitHub and has great support for different languages, data stores, and testing frameworks. Snap deploys your application to cloud services like Heroku, Digital Ocean, AWS, and many more. Try Snap for free. Sign up at SnapCI.com/RubyRogues.]**[This episode is sponsored by DigitalOcean. DigitalOcean is the provider I use to host all of my creations. All the shows are hosted there along with any other projects I come up with. Their user interface is simple and easy to use. Their support is excellent and their VPS’s are backed on Solid State Drives and are fast and responsive. Check them out at DigitalOcean.com. If you use the code RubyRogues, you’ll get a $10 credit.] **CHUCK: Hey everybody and welcome to episode 222 of the Ruby Rogues Podcast. This week on our panel we have Coraline Ada Ehmke. CORALINE: Hello. CHUCK: Saron Yitbarek. SARON: Hey, everybody. CHUCK: I'm Charles Max Wood from DevChat.tv. AngularRemoteConf.com. That's all I have to say. We also have a special guest this week. That's Sean Griffin. SEAN: Hello. CHUCK: Do you want to introduce yourself? SEAN: Yeah. My name is Sean. I'm a committer on Rails. I'm mostly known for maintaining Active Record. And I'm a developer at thoughtbot in Denver. CHUCK: Cool. So, does thoughtbot actually pay you to work on Rails or is that your own part-time thing? SEAN: It's been on and off thing in the past. So, I've had a couple of months of full-time but at the moment, no. It's just sort of a nights and weekends thing. CHUCK: So, we're getting real close now to having Rails 5 drop, aren't we? SEAN: Yeah. We don't have a specific date that we're shooting for yet. We've been shooting for August up until now but it'll probably be early September when the first beta goes out. It's mainly just Action Cable needs to get figured out. CHUCK: Yeah, I've been eying that. It looks really interesting. So, do you want to give us an overview of what's coming in Rails 5? Is it more of the same or...? SEAN: Yeah. Well, so the big headline features are definitely Action Cable and Turbolinks 3. And then we've been doing a lot of work on a lot of just smaller quality of life improvements. So, in Active Record for example we introduced a new API for type attributes which hopefully will make its way up to Active Model as there's time to just work on the implementation for that. You can finally do an OR statement with Active Record Relation which is the thing that everybody has been asking for, for three years now. And then just tons of small things. So for example, callbacks. It always used to be you have to make sure you return the truthy value at the end of a before save callback or it would prevent your record from saving. So, we have changed that now to be you throw the symbol abort if you'd like to halt the callback chain. And so, there are just lots of little things like that, lots of improvements of stability, improvements in performance. CORALINE: Do you think that Rails 5 will be easier for new developers to adopt than Rails 4 was? SEAN: Oh, [sighs] that [chuckles] CHUCK: Yeah, that can of worms. Let's open that one. SEAN: Yeah. [Laughter] SEAN: So, are you guys familiar with Turing School? SARON: Yup. CORALINE: Yes. CHUCK: I've heard of it. SEAN: My wife actually just graduated and I spend a lot of time mentoring the students over there. And I think one of the biggest problems with Rails right now is that we focus really, really heavily on the experience for an experienced developer typing 'rails new' but we ignore the brand new developer typing 'rails new' and the experienced developer working on a legacy app. And of course the improving the experience for brand new developer is a lot harder one to tackle, especially without making breaking changes. So, I don't necessarily think that it's going to be easier to use in any huge ways. But there are a couple of small things. I've been trying to focus on making newer APIs that we introduce into Active Record more explicit and less magic, and in particular trying to make sure that they can't as easily be confused by newer developers for new syntax in Ruby. And then one of the other things that we just did was I noticed that a lot of new developers open up config routes and we have that giant comment which explains everything you can ever do with routes. And that makes the file seem really intimidating and they tend to be afraid to change it until an instructor says. “No, it's okay to change it.” So, we just deleted that comment with a link to the guides. Or replaced it with a link to the guides, rather. SARON: I'm interested to hear how you decided to make these decisions. Because this is software development. But it's also product development as well, right? Because they're your users and you have to figure out what the problems are and you propose a solution and then you see how they react to it. What was that process like to figure out what changes to make, getting [inaudible]? SEAN: In terms of things for newer developers? SARON: Correct. SEAN: I guess it's specifically looking for common patterns in things that are hang-ups for new developers. SARON: Is that doing a lot of one-on-one interviews? Is that a survey? Like how did you even get to the point where you said, “I noticed that these are the top three issues that new developers deal with.” SEAN: I just spend a lot of time mentoring them. Basically my regular schedule is on Saturdays I spend the entire day mentoring somewhere between five to nine of them every week. So, I get a lot of exposure to where they're at across that entire program in particular. CHUCK: Are there any things that make you want to pull your hair out in frustration? SARON: [Laughs] SEAN: [Laughs] The usage of the word 'syntax'. SARON: Really? [Laughs] SEAN: I think it's important to understand that Rails doesn't add syntax, nor do we have the ability to add syntax. And it's not... when you're working with Active Record it's not 'you need to remember the syntax of Active Record', it's the API. And I think understanding that has_many is just a regular Ruby method which then means that because there's no dot in front of it, it's being called on the implicit self which means you have to be able to understand what the value of self is on every line... and ultimately you're not going to go look at how has_many is actually implemented. But I do think understanding that it is just a regular Ruby method, you can do regular Ruby stuff there, is key to them understanding what they're trying to accomplish in their usage of Rails versus just trying to write the code that works. CORALINE: You mentioned earlier about adding OR to Active Record. And I have the feeling that's going to be really big for new people too, because otherwise the only way to implement that was to drop down into raw SQL. SEAN: Right. Now that said, I actually don't think having to drop into raw SQL is necessarily bad. But definitely OR was one of those main pain points where you really wanted the composability of having the left side on its own and having the right side on its own and just being able to compose the two together. And so, the API that we landed on does not take a hash or anything like that. It's specifically a 'relation dot OR other relation'. So, if you're just trying to in-line a couple of where's and combine them with OR it's going to look really ugly, because it would be where dot OR where other thing. But it will look really nice if you're doing something like 'recent dot OR star' where you're using named scopes on those sides. CORALINE: That makes sense. CHUCK: I have to wonder, though. When you said that you were implementing OR, did anyone propose marriage? Because seriously, I get so tired of hammering it in myself. Because I mean seriously, it's such a pain to drop to SQL for something as simple as OR. If I have some funky condition that can only be expressed one way or another, then that's one thing. But OR just seems like it should be there. SEAN: Yeah. Ultimately, most of the work on that one was done by Matthew Draper. And then I just finished up the implementation after it had been sitting around and we had come to enough of a consensus to merge it. And I wanted to sneak it in before we started arguing again. [Chuckles] CHUCK: Boy, you would make a terrible manager. Matthew Draper was kind of involved. SARON: [Laughs] CHUCK: [Laughs] I'm just saying. [Laughs] SEAN: Sorry, yeah. That's a terrible way to phrase it. CHUCK: I know. SEAN: No, yeah. The credit for a fork definitely goes to him. CHUCK: I know. I'm just saying most of the managers that I've had, it was, “Yeah, Chuck helped.” [Laughter] CHUCK: So, I really want to dig in, because you sent us a couple of links to the APIs for attributes. SEAN: Yeah. CHUCK: And my first thought, and this goes back to the question that I think Coraline asked initially, was approachability for new programmers. In this case though, you said you wanted it to feel less magical, but it also feels more verbose. There's more crap I got to worry about instead of just being able to lean on the Rails magic and know that it's going to look at the database and do the right thing. SEAN: Right. Well, so this is one thing that's important to know about the Attributes API is that it leaning on the database like that all still is there. And in fact, the key to the way this is all implemented is now... and I call specifically when we look at the database and get all the type information that way 'automatic schema inference'. And that's now a single five-line method in a file called model schema. It's called 'load schema bang'. And all it does is grab the columns hash from the connection adapter, loop through it, and call the Attributes API for you. CHUCK: Oh, now that's slick. SEAN: Yeah. This is specifically giving you the power to hook into an override, something that we're already doing for you but now it has a concrete representation. Now, the API that's in the docs is slightly different than the one I wanted to ship. And you can still use it the other way. But basically, so in the docs you pass in a symbol to refer to the type. And if you want to add your own types you can register that in a global registry. The way I had wanted to ship it was where you always pass a type object. So, that means that when you see 'type string dot new', if you want to know the behavior of the string type you're not going to have to go figure out where this symbol in this context is documented. You have a specific greppable token. You have an object. It has specific methods you can call on it. You can find that object in the docs. And you can infer a lot of what behavior can be modified by this API based on what methods exist on the only object that you're passing in. CHUCK: Because it makes a lot of sense, so you can basically set up the attributes calls on just the things that you care about. And it looks like you can just override whatever it pulled in before. So, if I call attribute against name, maybe that's not a good example. But phone number, for example, and I want some kind of type casting or something on phone number or I have some other behavior that I need out of it... SEAN: Yup. CHUCK: Then it just ignores the old attribute setup and does the new one, the one that I called in. SEAN: Yeah. And that's exactly what the design is for, is that it is where if you want to use domain objects to represent these. And it is subtly different as well than just overriding a reader or a writer. There's a lot of other implicit behavior in our attributes system that can get broken if you don't implement a custom reader or writer a specific way. But the other important thing is now the types on the classes are the universal source of type information. So, anything that needs to know about how to do coercions has to go through this. So, that means that when you use the Attributes API to wrap the phone number column in your phone number object, you're not just getting the ability to use your phone number object when you're reading and writing from your model. You can also pass your phone number object to 'where'. CHUCK: Right. CORALINE: Does the end user have to worry about serialization for the writes to the database? SEAN: No. And in fact, I'm still trying to figure out if you do need specific control over how it goes to SQL or quoting behavior, where that should live. The contract here is that the type needs to know how to convert from the domain object to a reasonable Ruby primitive that the database adapter would reasonably know how to handle. CORALINE: But is that automatic or is that something that you as a developer have to write? SEAN: It would just depend on your needs. It would be automatic... I mean yeah, I guess it would never really be automatic other than if for example the underlying database type is a string type. We'll probably end up calling to_s on your object automatically. But if that's not sufficient then yeah, that's something you'd have to worry about. And I don't think that's necessarily a bad thing. A lot of types are just going to have the one generic 'handle everything and do this one transition'. Like the integer type in Active Record for example doesn't have specific control over each direction. I do want to make sure that the hooks are there so that if you do need more control, you're not having to reach into the internals. CORALINE: I think it's a really good thing. And it reminds me of the way schemas work in Mongoid. You can have custom objects as long as they serialize down to something the database can understand. You can define your columns as being an object type that you've created yourself. So, I think that's a really great thing. SEAN: Yeah. And it's definitely not a complete solution for everything database-related. But I have been trying to keep the database aspect of it semi-separated from this API in particular just because I'm intending to move it up to Active Model. CHUCK: Oh, that would be cool. It looks like you can actually create different custom types. And I'm kind of curious. So, it has the money type in here. And I got a little bit confused when I was looking at how it worked. So, you have the money type serialize and then you've also got a money type that's just a value type that does a typecasting. SEAN: Okay, yeah. So, you're looking at the money type example. So, you create your class. CHUCK: Right. SEAN: And type objects have three methods: cast, serialize and deserialize. CHUCK: Okay. SEAN: So, serialize and deserialize are transitions from database value to Ruby value and back. And then cast is from user input to Ruby value and user input can either be a submission from the form builder or it can be just an assignment in Ruby land. CHUCK: Okay. So, if I want something that casts into say an integer which is what you've got in your example here, then I just have it inherit from Active Record type integer and then set up cast so that it returns an integer? SEAN: Right. Or in this case, we're calling super here and letting the integer type do all of its stuff. And we're just doing things like stripping out the dollar sign. CHUCK: Right. And then it looks like you've also got another one here that actually inherits from type value. So, what's the difference between type value and Active Record type integer? SEAN: So, value is just sort of the superclass of all of the types. You don't have to inherit from it. The API is pretty small. It's just there for convenience, because one of the common patterns I noticed was that a lot of types just needed to implement cast. And for a lot of the types, the logic for deserialization from the database was exactly the same as the logic for taking input from the form builder. So, it just does a little, a couple of convenience things like aliasing deserialize to cast, by default. Cast is actually implemented as calling a method called cast_value, which is not part of this object's public API. But what you can do is if you just don't want to handle nil, you can inherit from type value and define cast_value and then it'll just filter out nil for you. So that way, you don't have to have the 'return if value nil' at the top of every type, ever. It's just a superclass that adds a couple of convenience methods CHUCK: Ah, gotcha. SEAN: It used to be called type identity in an earlier version of this API because if you were to actually pass this as a type object to the Attributes API it would just always be the identity type. So, it would always return whatever was given to it. CHUCK: So, I guess the other question is when would you use something like this over say, creating just some convenience method that does the conversion or serialization for you? SEAN: Pretty much always, because you can actually cause some subtle breakage in terms of dirty checking, working with the form builder. But really, if you're not concerned about that which I don't think most people are, you would use this. When you want to be able to differentiate between values that came from the database versus values that came from the form builder, or if you'd like to be able to pass your domain object to 'where'. CHUCK: Oh, gotcha. And that's pretty convenient. So, this all integrates nicely with all of the other finders and query methods so that you can... I guess it's basically just 'where'. But you can pass in a formatted value and it'll do all the clean-up and serialization/deserialization for you. SEAN: Right. Well, you can either pass in user input directly or the other case being if you... I've tried to maintain a contract of if you read a value out of an attribute on a record and you pass that to 'where', you should get that record back. So, there should always be reflexivity between those two. CHUCK: So, let's use the example here for a minute of phone numbers. Some people just use dashes. Some people in the US anyway use parentheses around the area code. Other countries format them different ways. Some people use dots. So, I can create something that serializes and deserializes one string into a common format. And then when I do the 'where' lookup, I can pass it with any of those formats and it does the serialization properly there, too. SEAN: Yes. CHUCK: Okay. SARON: That is so awesome. SEAN: Yeah. And it's important to note that it seems like a semi-simple API on the surface. But it actually required rewriting half of the internals of Active Record. But this is the universal source of type information. Everything else in Active Record that deals with type information goes through this API. So, it's not even just... it really is just 'where' at the end of the day. But anywhere in Active Record that type coercion occurs, you can be assured it will be going through the information that you've given us in the Attributes API. So, it will never accidentally fall back to the database schema. CORALINE: That sounds like a massive amount of work. How long did take to implement that? SEAN: About a year, six months of which was full-time. CORALINE: Wow. SARON: Wow. Did you do it on your own? Or did you work at a team on that? SEAN: It was mostly on my own. Aaron would periodically have a call with me and we'd go over blockers. And Rafael of course was very helpful, because I was very new to the codebase when I started working on this. But yeah, it's just been a long process. CHUCK: Well, thanks to Aaron and thanks for thoughtbot for any time they paid for. SARON: Mmhmm. CORALINE: Yeah, it sounds like an amazing, amazing feature. SEAN: Yeah. And the internals that came out of it are just, like entire classes of bugs have disappeared in terms of... so, there's another internal API that's build on top of this that may be public at some point in the future. But it's called an attribute type decorator. And we actually it turns out have several places in Rails today that modify the types, the most common one being the serialize macro. There is also a thing that is turned on by default that I still can't remember off the top of my head exactly what it does and I don't think most people are aware it exists. But it's called timezone-aware attributes that modifies the behavior of certain types. And then pessimistic locking for some reason modifies the behavior of one of the types. But we actually had a bunch of subtle bugs where that information would live on the Active Record class but not on the database column object. And if anything ever went and got information from the wrong place, it would miss out on this behavior and you would get weird thing like your object that was serialized with YAML actually ends up going to the database as a YAML-dumped string of YAML. And so then, when it deserializes back out, you've got a string of YAML. And stuff like that. And so, those kinds of bugs just completely went away. And then a lot of the more subtle errors that we had in certain types also, it's not that they're impossible to write now but they're much less likely to occur when we now have this one object whose only responsibility is to deal with this specific data type and that's all it deals with. And so, the errors in that behavior become much more obvious. CORALINE: I'm really curious about the process behind a feature like that. Do you go to someone and propose, “This is what I want to do”? Or do you spend that year and hope that it gets merged? SEAN: Definitely propose it first. [Chuckles] Yeah, so if anybody else has a feature that they would like to spend time working on but they're not sure if it would be accepted, the Rails Core mailing list is the place to propose it. In my case, Aaron had come to Denver to speak at a local meetup and we went to a bar afterwards. And I got really drunk and shouted about how this should be in Rails. SARON: [Laughs] SEAN: And then he said, “Okay, go do it.” [Laughter] CORALINE: The transcription [report] lives on the Rails Core mailing list. SARON: [Laughs] SEAN: Yes. CHUCK: Yeah. SARON: So, that's the secret. Good to know. SEAN: Yes. The secret is get drunk and yell at Aaron. SARON: Yup. I like it. SEAN: And we have policies which are documented but they're in this giant contributing guide that nobody actually reads the whole thing. And so, this tends to get missed a lot. So, our policy is that the issues tracker is only for specifically bugs. So, feature requests go to the Rails Core mailing list. If you would like feedback on an idea before trying to implement, [inaudible] the Rails Core mailing list. And then if people need help with something, the Rails Talk mailing list or Stack Overflow are the right places. But we have a specific hierarchy. CHUCK: Very interesting. I got a couple more questions just looking at the API here. And these may have been in Rails 4 and I just didn't see them. The first one is I see eager_load, includes, and preload. What is the difference between those? SEAN: Those are in Rails 4. So, includes is specifically saying that you are joining onto this table and you'd also like to eager_load it. And I don't remember what the difference for eager_load and preload are [chuckles] to be perfectly honest. I know that they get referenced differently by our join dependency class. But I honestly couldn't tell you when to use eager_load. I just always use includes. CHUCK: Yeah. I usually do, too. I was just wondering if there was an advantage to using the others. SARON: Yeah. I remember doing having to use include for the first time a while ago. And there's a pretty good blog post that talks about the difference between includes, preload, and eager_load. So, I'll put that in the show notes and that'll help people out. SEAN: Cool. CORALINE: Is that going to lead to less monkey-patching? SEAN: The Attributes API? CORALINE: No, I'm sorry, the preloading. SEAN: Oh. Yeah, I'm not sure. Like I said, these are all in Rails 4. CORALINE: I'm sorry. I think I'm confused. I was talking about the model prepend which I guess is a different topic. SEAN: Oh, yes. CHUCK: Go for it. Model prepend. I don't even know what that is. CORALINE: It's a new feature in Ruby 2.2. CHUCK: Oh, okay. CORALINE: And Rails 5 is going to support 2.2 and up only. SEAN: Yeah. And we do reject any changes that are just specifically like, “This code can be written a little bit better,” because we're on Ruby 2.2. But as we are already modifying code or if there's performance benefits we've been taking advantage of a lot of benefits. The main reason for us targeting 2.2 only is symbol garbage collection. CHUCK: Oh, that's interesting. So, it's a performance choice? SEAN: Well, it's always been a security problem for Rails if we allow... CHUCK: Oh, yeah. SEAN: If we use symbols anywhere internally, it could potentially cause a DOS vulnerability. And now, that's no longer a concern. CHUCK: Gotcha. Yeah, and I knew about the Ruby prepend but I never really considered how it would apply to models. That's an interesting avenue to explore. We should put a link to that in the show notes as well, to a blog post or some documentation. SEAN: Yeah, I'm trying to think where we're using prepend internally. I think we've refactored some of Action Support to benefit from it. There were a couple of places where we were having to do exactly what you did before model prepend where we grabbed unbound instance methods from the thing that we were prepending onto and then use that as super internally, which is hacky and awful and is what makes our codebase look so terrifying when people look at it for the first time. SARON: [Chuckles] CHUCK: So, I want to ask another question. This gets more into the internals of Active Record. But I was trying to think about how you would implement something like OR. Do you want to talk about how you deconstruct the APIs and then construct SQL queries from them? SEAN: Yeah. So, OR is pretty straightforward, actually. So, one of the big concerns with OR as well is that it's actually a very ambiguous method because OR doesn't only exist in where clauses. It can also exist in having clauses. And several people actually think that just the method relation dot OR implies a union, a SQL union, and not a SQL OR clause. So, the first thing that we do is we compare the two relations to make sure that they are structurally compatible, which just means that the only place they differ is in either the where clause or the having clause. And internally now there is a single class called where clause which actually having clause is just an instance of where clause as well, because they follow the same semantics in terms of SQL. And then we walk the AST that we've got for the existing, the Arel AST that we've got for the existing where clause, look for any branches of that tree that are identical so that we can dedup them. And then at that point, it's just you group up what's left on the left side and on the right side, put parentheses around it, stick an OR statement in the middle. CHUCK: Gotcha. So, yeah, so just pulls it apart, builds an AST, and then puts it back together? SEAN: Yes. CHUCK: And so, inserting an OR is just a matter of adding another node type in your AST? SEAN: Well, Arel already has an OR node, because Arel has a more or less complete AST representation of SQL. CHUCK: And just to back up. I know we have some newer listeners. An AST is an abstract syntax tree. SEAN: Yes. CHUCK: And basically, the way that it works is if you look at a query statement like 'post where id is 1 dot or post where id equals 2', what it does is it creates a tree. So, the top of that's going to be the OR. And then underneath it it'll have an equals and then it'll have an id on one side and a 2 on the other. And this is just an example. I don't know how Arel puts it together. But anyway, it's a way of representing complex sets of logic so that you can then put it back together in a different way. And in this case, what it does is it creates SQL which is the language that's spoken by the database. SEAN: Yes. Yeah, and the reason that we specifically need to use an AST in our case is because we want to have our code written in a generic way. But we might need to actually produce very different SQL depending on if it were going for MySQL, PostgreSQL, or SQLite3. CHUCK: Oh, that makes sense. SEAN: And I'd also just like to mention since we're talking about Arel, Arel is a private internal thing. And it breaks a lot between versions without deprecation. Please don't use it. [Chuckles] CHUCK: Gotcha. I'm also wondering looking at this API document, what are bound attributes or what does the bound_attributes method do? SEAN: Is that listed in the public API? CHUCK: It is. SEAN: That should not be. SARON: [Chuckles] SEAN: There was a thing called bind values on relation. And it was bad for a lot of reasons. And this is a thing that internally represents it differently. I named it differently because I used to have one implemented in terms of the other so that I didn't have to in one commit go change everything in Active Record to use this new structure. But basically internally, it represents, whenever you see a query that has a placeholder. So, when you do 'where id colon 1', the actual query that gets run will be 'where id equals question mark on MySQL or dollar sign 1 on PostgreSQL'. And then the one gets passed in separately. And that's a prepared statement. And there are a lot of benefits to using those, the biggest one being that the database doesn't need to rebuild the query plan over and over again. And then we're able to also do caching on construction of the SQL query on our side. But ultimately then, we need to separate out, because now in our AST that represents the query, what we actually have is a placeholder for the value and we still need to pass the value separately. And so, this is the thing that... this is where the actual values get held onto so that it can get sent to the database later. And it's called bound_attributes because it uses an attribute object internally, which is the same object that we use to represent an attribute on your Active Record model. Or more generically, anything that has a name, a type, and a value that needs to get cast by that type. SARON: I'm wondering. How often do you come up with an idea or something you want to build and add to Rails? And then you try it out and it either ends up not working for whatever reason or just being a really bad idea. SEAN: Like, 70% of the time. [Chuckles] SARON: Really? SEAN: Yes. SARON: Wow. That's a pretty high percentage. Why do you think that is? SEAN: A lot of the reasons when I try and do something and it doesn't quite work, it's not necessarily because it's a bad idea. It's just because the codebase is [inaudible] it. So for example, one of the things that I probably tried 20 some odd times to do before it was actually possible, feasible, was getting the Attributes API implemented in that way where the automatic schema inference just calls into it. And a lot of it's just because there's a lot more refactoring that needs to get done before I can structure the codebase a certain way. And I don't like introducing code into the codebase that I'm not proud of or wouldn't want to maintain. So, if it ever just doesn't go the way I would like it to, I scrap it. SARON: At what point do you decide not to continue with the feature? Is there a magic number of times tried before you let it go? SEAN: I don't think, no, that I have a concrete “Here's when I just abandon it and don't ever come back to it.” It's always just really gut feeling as to whether it'll work or not work. CHUCK: So, you get to a certain threshold of pain or of inconvenience. SARON: [Chuckles] CHUCK: Or of, “Gee, this is really ugly.” SEAN: Yeah. But that's the thing, right? Because the Attributes API spent over the course of a year dozens of times trying, “Are we ready yet? Are we ready yet?” and stuck with it. And then there have been other times where there was an API... and I still do want to support this general concept but the specific API I had in mind for how to deal with composing attributes together, I abandoned that after about a month. So yeah, it really is just gut instinct I think. Going about developing Rails I guess is more or less how normal developers go about developing normal things. Because we are ultimately just as bunch of normal developers and Rails is ultimately just a really large legacy codebase. CORALINE: Speaking of legacy codebases, you said at the beginning that there were two primary audiences, if I understood you correctly: brand new developers and then developers who are experienced who are working on legacy codebases. And you mentioned making things easier for the latter population. How does Rails 5 address the issue of legacy code? SEAN: So, it's not so much about addressing the issue of legacy code. It's mainly about making sure that we provide the level of control that a larger or older app tends to need. So, my rough litmus test for this is if there's a person with some sort of app and they are a reasonable developer who makes reasonable design choices and they're still having to monkey-patch or reach into internals to accomplish whatever they need to get done for their business, I feel that is a bug and that we are missing some hook for them. Did you guys watch Yehuda's keynote at RailsConf two years ago? CHUCK: No. SEAN: One of the things he talked about was building upon shared abstractions and how ultimately software is like this skyscraper and we're on the 200^th floor. And his talk is about how we all just need to get on the 200^th floor and accept that the 200^th floor exists. And then Ernie Miller that same year gave a talk about how that's all cool, but presumably the 195^th floor exists for reasons other than just supporting the floors above it and there might be cool stuff there that you want to visit. And I think a big part of it is just making sure that those lower floors, those lower levels are still there and are pleasant to use and are implemented in such a way that you can modify a behavior in a reasonable way without having to monkey-patch or reach into internals. CORALINE: How much time did you spend thinking about the migration path from Rails 4 to Rails 5? SEAN: Every [new] change. Rails 5 does have a small handful of breaking changes that did not have a deprecation notice because it is technically a major version. But those should impact almost no users. But we try very hard to make sure that if you are only using public API and you have fixed your deprecation warnings since the last year, then the upgrade should be completely painless and it should just work. CHUCK: Well, and in my experience when you go from one major version to the next, like from 3 to 4 or 4 to 5 or even, what was it, 3.1 to 3.2 was kind of a big thing, you give us another minor version update that has the rest of the deprecation warnings in there so that you can clean up anything, even these edge cases that aren't going to affect too many people. SEAN: Right, yeah. And then the main times that we'll make that kind of change is if it is some case that we really do feel strongly is a change that needs to be made. But we cannot either due to performance reasons or just inability to actually detect if you're relying on the old behavior, if for whatever reason we cannot actually give you a deprecation. CHUCK: Yeah. SEAN: And I can't actually even think of off the top of my head what those cases are. CHUCK: So, we've talked a lot about active record. Are there other changes in other areas like Action Controller or Action View or Active Model which is tied into Active Record, or any of these other tools that we use? SEAN: Yeah. Active Model got a couple of minor quality of life improvements, just things that people tend to want. A lot of it was just methods that used to exist on Active Record have been moved to Active Model. Action Controller and Action View, there must have been something user-facing that's changed, presumably bug-... there have definitely been plenty of bug fixes. I can't think of any. Oh, deep munge is going. That's in 5.1 I think. But that'll be huge. So, okay. We have this issue where in Active Record we need to make sure that we don't ever allow nil to accidentally get into a query. And that's because for certain configurations of MySQL if you do 'where primary key is null' it will dump the entire database. And people still use it with this configuration. And we can't force them not to. CORALINE: That's frowned upon, right? CHUCK: [Laughs] SEAN: Yes. CHUCK: I sure hope so. SEAN: Yes, it is. But we can't have it be insecure for this really common case. And so, there was this issue where because of the way 'where' worked, if you passed an array containing just nil, that would do 'where thing is null'. And you might be in your code, if you are using this version of MySQL, of course if you pass us nil explicitly, then we're going to do what you told us to. But it's more like empty string wouldn't cast to nil and stuff like that. And so, a lot of developers who were in that scenario before they would call find, they would do 'if params id' or a 'must params id dot nil'. But if the params sent is JSON where you can send and array containing nil, that would not be null, or that would not be nil in Ruby but it would be treated as nil when it got to Active Record. So, a thing called deep munge was introduced in some version of Rails 3 where if an array is containing only nil, basically yeah if an array contains only nil we treat that as nil. And as a side-effect of that we also treat empty arrays as nil. And I believe we also just remove nil from arrays in general, which is fine if you aren't making a JSON API. But if you're making a JSON API it can lead to some really confusing behaviors. And what we decided to just end up doing was make 'where' a little bit dumber. And so, there'll be a deprecation cycle in Rails 5 for this. If you pass an array containing nil to 'where' then it's going to in Rails 5.1 generate the query 'where value in null' which would never return a value. But we're just going to do exactly what you told us to. And if you want to check where it is nil specifically, then you can just pass nil, not an array. Or if you want to pass in an array of values and you want to do 'or the thing is null', then now with relation OR that's a thing that you can do. But I think for the common case, most developers have never really explicitly tried to use 'where' to be like 'where this value's 1, 2, 3 or null'. So, that'll be a thing that affects Action Controller. But I don't think there are many huge changes in Rails 5. CHUCK: Yeah, I didn't see too many. It seemed like most of it was what we've talked about and Action Cable, which I think deserves its own show. SEAN: Yeah, definitely. The other thing to keep in mind too is that the Rails codebase is 270,000 lines of code. And 210,000 of those are Active Record. So, the other gems comparatively do [inaudible] time. Rack 2 is still coming along. And that will definitely be something that affects the controller layer. Although I don't think that most apps are going to actually be affected by it just because adding HTTP 2 support, there are very few user-facing pieces of that, other than server push where the server is able to, when you ask for one file send and you're going to ask me for 20 other files [inaudible] are right now, and that will have an explicit API at some point. But that will be Rails 5.1. CHUCK: I guess the other question I have related to what we've talked about with Active Record is I remember from Rails 2 to Rails 3 a lot of people were upset because it took a major performance hit. How does performance of Rails 5 compare to Rails 4? SEAN: It's faster. CHUCK: Okay. SEAN: Faster than 4.2. Yeah, so we've got long-running benchmarks now. Sam Saffron worked with... I can never remember how to pronounce his name. So, worked with a guy who's tgxworld on GitHub and then also with Kir Shatrov to get us the Ruby bench project and to add Rails support to it. So, we do have long-running benchmarks now that can tell us on a per-commit basis changes in some simple micro-benchmarks. And then Discourse focuses very heavily on performance. So, they also serve as a macro-benchmark for us. So, we have much better visibility into performance. CORALINE: I know that Aaron has worked on performance quite a bit. The one time I paired with him on Rails code, it was specifically to improve benchmarks. SEAN: Yeah. There are really only three things that you can work on in Rails. You add a new feature, you refactor to make the code more maintainable or you try and improve the performance. So, we see a good bit of all three of those. And yeah, I think just at this point we're all on the same page that any performance regressions between versions is a bug. CHUCK: That's interesting. CORALINE: Nice. SEAN: And I mean of course there's minor exceptions like if the performance got worse in some case because it got better in some more common case of doing the same thing. CHUCK: Yeah, but that's overall in my opinion a performance gain across the different Rails apps out there. SEAN: Yeah. It's just one of those things though that also makes gauging this difficult especially since one specific benchmark might look abysmally bad. CHUCK: So, I really have been thinking lately about the podcasts and I want to make them more actionable. I think we've learned a lot of things about Rails and it's interesting to dig into a lot of this stuff. But if people want to go try this, if they want to go see what we're talking about, they can go look at the documentation. But if they really want to go try it, do they just go and in Bundler add the Git repo for Rails to their Bundler and then just go for it? SEAN: Yup. That's exactly what you can do. Secret little thing is that the Attributes API is actually in Rails 4.2. It's just not public because there were implementation details I was not happy with. Although your mileage may vary and the actual API looks a little bit different in 4.2. But you can actually try it out on 4.2 but your mileage may vary. And it's internal. So, if it breaks your app, sorry, but that's not a bug. [Chuckles] And then yeah, the biggest thing that most users can do is please, please, please when we do put out the beta, try the beta. Just bump your gem version, run your test suite, see if something breaks. Or if you notice that migrating is painful for you and it's for a reason other than you were using internals, you might have some use case that we didn't consider. And we can still fix that during the beta period. But once Rails 5 ships, if we didn't consider your use case, now considering it might be a breaking change, and sorry. So, please, please, please try the beta. CHUCK: Yeah. So, try the beta means plug it in, run your tests. SEAN: Do not deploy it. [Chuckles] CHUCK: Right. Stand up your app, see if that works. SEAN: Yeah. Maybe you run it in staging. We're at the point where master should be pretty stable. If you have CI maybe put running against Rails master at a loud failure in your CI matrix. CHUCK: I guess the other question I have is are there things that people are likely to run into with the current codebase or certain areas that are maybe less baked than others that are more prone to have bugs in them? SEAN: Action Cable. But that's just because it's new and doesn't have a lot of tests at the moment. But hopefully it will be stable in Rails 5. But yeah, that will most likely be the most unstable part of the codebase for the time being. CORALINE: You have no idea the restraint I'm showing right now to not respond to that testing comment. CHUCK: Which one? CORALINE: That it doesn't have a lot of tests around it. CHUCK: [Chuckles] SEAN: Yeah. Anyway. [Chuckles] CHUCK: [Laughs] SEAN: Yes. I think we are on the same page on that one. CHUCK: I was holding my tongue trying to goad somebody into saying something. CORALINE: It worked. CHUCK: [Chuckles] SEAN: Yeah, we'll see. The majority of the Rails teams of course has not... we were not able to see it or work on it before you guys were able to see it. So, this is all new to us and we're just trying to get our heads around the codebase that we're going to be maintaining. CHUCK: Yeah. It's being lifted directly out of Basecamp from what I understand. SEAN: Yes. It was very much a Basecamp production. But it will have tests by the time Rails 5 ships. That much is certain. CORALINE: Good. CHUCK: Are all of the Rails guides and everything else up-to-date for edge Rails? Edge Rails is for those unfamiliar with the term, is whatever is in master on the Rails repo. SEAN: Yes and no. Yes in that if there was a specific breaking change, the guides should be up to date. But no like they're not exhausted and they might be missing information about new features or better ways to do things. As an aside, if you're listening and you're interested in getting involved in Rails, that's a really, really good way to get started, is if you see something in the guides or the documentation that is missing, confusing, could be worded better, could be improved, we always love to get more people helping with that. CORALINE: There's a separate section in the contributor guide to how to contribute to documentation which I really appreciate. SEAN: Yeah. And if you don't want to open a pull request, you can just open a pull request, you can also just ask for commit access to docrails which is a mirror of Rails but we give anybody who asks commit access to it. The one restriction there is that you are not allowed to change code. Although these days, I think most people just open a pull request because it's easier. CHUCK: That makes sense. Alright then. Before we get to the picks, I just want to acknowledge our silver sponsors.**[This episode is sponsored by Code School. Code School is an online learning destination for existing and aspiring developers that teaches through entertaining content. They provide immersive video lessons with in-browser challenges which means that each course has a unique theme and storyline and feels much more like a game. Whether you've been programming for a long time or have only just begun, Code School has something for everyone. You can master Ruby on Rails or JavaScript as well as Git, HTML, CSS, and iOS. And more than a million people around the world use Code School to improve their development skills by learning or doing. You can find more information at CodeSchool.com/RubyRogues.]****CHUCK: Saron, do you have some picks for us? SARON: Yes, I do. I have a couple. One is an article that I think came out two days ago called 'Design sprints: what are they for?'. And it's just really well-written. It's super concise. It goes through first of all this idea [inaudible] design sprint and a discovery process. And it goes through what to do on each day and how to break it down. And I've seen lots of design sprint explanations. Thoughbot has a really good one as well. But this one was just really, really short, concise, and to the point and I really liked it. So, this Saturday I keynoted LoneStarRuby Conf and it was awesome. I got to meet Avdi in real life for the first time. I want to report that his voice and his hair and just as flawless as they are online. So, that was exciting. And he gave a talk called 'The Soul of Software' that I'm so excited for everyone to hear when it comes out. The whole conference was recorded and it's going to be on Confreaks soon. So, just keep an eye out for 'The Soul of Software'. It's an absolutely beautiful really, really deep, thoughtful talk. And I just wanted to share that with everyone. And my last one is, so CodeNewbie we finally gave in and set up a Patreon account. So, if you're interested in supporting our work and the podcast and the Twitter chat and all things that we do, you can go to Patreon.com/CodeNewbie and support what we do. So, those are my picks.CHUCK: Alright. Coraline, do you have some picks for us? CORALINE: I have a couple of picks today. The first is something called Mockaroo. It's a website tool that lets you generate realistic test data in CSV, JSON, or SQL. You can basically set it up with whatever fields you want to generate. And your options include cities and colors, credit card numbers, boolean values, domain names, Chinese family names, color, latitude and longitude. The list is super long. One of the fields called naughty string even generates data that's often problematic when you bring it into your code, like multi-byte characters. And you can even create formulas or regexes to compute a field value based on data from other fields. So, it's pretty useful when you need to create a lot of test data and run that against your code in fixtures or what have you. The second pick is the art of Jim Kazanjian. And I'm probably massacring his last name. What he does is he uses images from the Library of Congress photo archives and assembles what look like photographs of sinister buildings. A piece is typically made from over 50 distinct photographs from the last century. He's inspired by works of classic horror like those from author H.P. Lovecraft. I found his art to be a great source of desktop images. That's how I appreciate a lot of art. One of my favorites is a great Victorian house on wheels. It reminds me of 'Howl's Moving Castle'. So, check out his art. It's really amazing, really beautiful and really surreal. CHUCK: Very cool. I'm going to go ahead and do... occasionally I do picks that aren't actual things you can go buy and look at. In this case I've been working on Angular Remote Conf for a while. And one pick that I have is mastermind groups. And I know I've talked about them on this show before. But I was talking to my mastermind group about what I wanted to get out of the conference and who I wanted to be there and who I wanted to attend and things like that. And they called me out and basically told me to make a plan. And so, I made the plan and then they said, “Well, now you've got a lot of things in your plan. So, go put them in your calendar so you don't get overwhelmed.” And so, I'm just going to pick basically making a plan, planning out the implementation and then executing. It's been really less stressful to just know, “Okay, I'm going to do this at this time and this at this time. I'm going to reach out to these people for this and trying to get this to happen in these ways.” And anyway, it's been a really, really positive experience for me. So, those are my picks. I'll probably pick some of the tools that I'm using next time. But for now if there's some major or big outcome that you want to get, then sit down and plan it out, and then figure it out. And yeah sure, not everything's going to go to plan. But at least that way you know what the next right thing to do is. Sean, do you have some picks for us? SEAN: Yeah. I'm just going to have one pick. I'm going to pick the Rust programming language, which I'm going to guess has probably been picked before. But I've been working on a lot of C++ code doing some augmented reality stuff. And I ported it to Rust in my free time. And I fixed a bug in the C++ code because the Rust version with that bug wouldn't compile. And it's a really cool language. There's a lot of Rubyists checking it out right now. There's even a book 'Rust for Rubyists' that Steve Klabnik wrote. And yeah, people should check it out. CHUCK: Yeah. And I'm pretty sure we did an episode on Rust. So, I'll find that and put a link to that in the show notes as well. If people want to follow anything else that you're doing or see what's going on with Rails core, what are the best ways to do that, Sean? SEAN: Well, I also have a podcast called The Bike Shed that we do every week now. So, you can find me on that. It's Bikeshed.fm. I'm also going to be giving a talk on the Attributes API at RubyConf Portugal and at WindyCityRails next month. I also gave an early, a shorter version of that same talk at RailsConf this year, which is on Confreaks. CHUCK: Alright, very cool. Well, thank you for coming, Sean. It was fun to talk and fun to explore what's coming up in Rails 5. SEAN: Yeah, thanks for having me on. It was great to talk to you guys. [Hosting and bandwidth provided by the Blue Box Group. Check them out at BlueBox.net.]**[Bandwidth for this segment is provided by CacheFly, the world’s fastest CDN. Deliver your content fast with CacheFly. Visit CacheFly.com to learn more.]**[Would you like to join a conversation with the Rogues and their guests? Want to support the show? We have a forum that allows you to join the conversation and support the show at the same time. You can sign up at RubyRogues.com/Parley.]**