043

043 AiA Falcor with Jafar Husain


Check out and sign up for Ruby Remote Conf!

02:09 – Jafar Husain Introduction

02:42 – Falcor

06:56 – MVC (Model View Controller)

  • Separation of Concerns

17:41 – Performance

34:23 – Angular 2

  • Asynchronous Binding
  • Data Access Patterns

Picks

Land of Lisp: Learn to Program in Lisp, One Game at a Time! by Conrad Barski, M.D. (Lukas)
Stevie Wonder: 1-2-3 Sesame Street (Lukas)
Visual Studio Code (Ward)
Traction: Get a Grip on Your Business by Gino Wickman (Chuck)
Wool by Hugh Howey (Chuck)
MediaWiki (Chuck)
PureScript (Jafar)

This episode is sponsored by

comments powered by Disqus

TRANSCRIPT

LUKAS: [Singing] Holla atcha boy…

[This episode is sponsored by Hired.com. Every week on Hired, they run an auction where over a thousand tech companies in San Francisco and New York and LA get on JavaScript developers providing to put the salary and equity upfront. The average JavaScript developer gets an average of 5-15 introductory offers and an average salary of over $130,000 a year. You just can either accept an offer and go right into interviewing with the company and neither with that any continuing obligations. It’s totally free for users, and when you’re hired, they’ll also give you a $2,000 signing bonus as a “Thank You” for using them. But if you use the Adventures in Angular link, you’ll get a $4,000 bonus instead. Finally, if you’re not looking for a job but know someone who is, you can refer them to Hired to get a $1,337 bonus if they accept the job. Go sign up at Hired.com/AdventuresinAngular.]

[Does your team need to master AngularJS? Oasis Digital offers Angular Boot Camp, a three-day, in-person workshop class for individuals or teams. Bring us to your site or send developers to ours — AngularBootCamp.com.]

[This episode is sponsored by Wijmo 5, a brand new generation of JavaScript Controls. A pretty amazing line of HTML5 and JavaScript products for enterprise application development. Wijmo 5 leverages ECMAScript 5 and each control ships with AngularJS directives. Check out the faster, lighter and more mobile Wijmo 5.]

[This episode is sponsored by Digital Ocean. Digital Ocean is the provider I use to host all of my creations. All the shows are hosted there, along with any other projects I come up with. Their user interface is simple and easy to use. Their support is excellent. And their VPSes are backed on solid-state drives and are fast and responsive. Check them out at DigitalOcean.com. If you use the code “angularadventures” you’ll get a $10 credit!]

CHUCK: Hey, everybody! And welcome to Episode 43 of the Adventures in Angular show. This week on our panel, we have Lukas Reubbelke.

LUKAS: Hello!

CHUCK: Ward Bell.

WARD: Howdy!

CHUCK: I’m Charles Maxwood from DevChat.tv. Real quick, I just want to let folks know that I am putting the other Ruby Remote Conf. If you want an online conference about Ruby, then check that out – RubyRemoteConf.com.

We also have a special guest this week, and that’s Jafar Husain.

JAFAR: Hi there!

CHUCK: Do you want to introduce yourself real quick?

JAFAR: Yeah! My name is Jafar Husain. I’m a tech lead at Netflix, and I’ve spent the last few years working on a software called Falcor, which is the data platform we use inside the Netflix and powers all the Netflix applications.

I’m also a representative on TC39, our Netflix is, and I frequently attend the meetings and working at the moment to get support for Reactive Programming and to better support Reactive Programming with the JavaScript language.

CHUCK: Oh, cool! I was going to prepare for this episode; I couldn’t find a repository for Falcor.

JAFAR: That is true. It’s because Falcor is not released yet. There’s some people, I think, out there in the community, they have mixed feelings about talking publicly about software that you planned open source before releasing it, and I totally understand that. But I think, part of our renaissance is that Netflix has a certain type of architecture and a certain type of design, and we would like to get more mileage out of working with internal partners without necessarily fully opening up to the community yet and having a stampede towards the solution that we don’t really necessarily feel has been validated enough outside of Netflix. We’re just like a little more bake time so we basically have an open-ish approach at the moment where people have requested early access to help do things like ports or builds and early software on it. And we basically want to sort of try and catch any issues that occur in the design problems they have when they tackle different types of applications in the Netflix before we kind of open it up to the world, and that’s partly to do with our resources. You know, Netflix does a lot of open source projects, but at least, I think this is our first big UI  open source projects, so it’s a different organization. And we just want to sort of leverage whatever resources we can out there in the community to help us make Falcor better before we necessarily open the flood gates. It’s just going to be easier for us to manage the feedback and sort of taking some cautious approach. But I totally understand that some people say, “Hey look, just open up the source code and that’ll get this done faster,” and that’s we’re just trying to take a balanced approach.

WARD: I was also looking for some kind of design docs or something like that, and maybe I don’t know how to find them or are those also not yet ready for primetime?

JAFAR: I wouldn’t say it’s so much that “they’re not ready for primetime”. I mean, that’s less of a desire to sort of keep those hidden. In fact, we were hoping to get API docs maybe out early sometime in the next week or a week and half or so. The hope was that the API docs can certainly get people a flavor of what it feels like to use Falcor. And we’ve done several presentations, and we talked about high-level design and the fundamental tradeoffs we’re making, which were really important to understand. I’m certainly happy to elaborate on those here as well if folks are interested in really what Falcor was trying to do from a 500-foot level.

CHUCK: Yeah, I think it would be useful to kind of talk about what it’s supposed to do and some of the principles behind it. Then, even if people can’t check out Falcor, they can go experiment with some of the approaches that you have there.

WARD: Even if you get there, can you start with what problem were you guys trying to solve with Falcor?

JAFAR: Yeah. Great question. Netflix is, fundamentally, a browsing problem. You got mostly static, at least, for the duration of an individual user. Netflix got a pretty large catalog of information that’s mostly static. And we want to run on all types of different devices. We want tremendous reach at Netflix. We run from the $15 DVD player to all the way up to the next-gen console, so we need to be able to run our applications in a whole range of different hardware platforms.

Those are two different challenges. The thing about mostly static data, it means that we can benefit a lot from caching, and I think that’s true with a lot of web applications out there. It’s mostly [incomprehensible]; you’ve got mostly static data with occasional rights. So in theory, you would benefit tremendously from caching. The challenge is that some of the network architectures and some distributor architectures that developers have available to them right now don’t really had to seem to hit the sweet spot, in our opinion, for those types of web applications. What I’m referring to specifically are REST and RPC. Usually when I talk to UI developers or group of UI developers, I don’t think they start on the network side because, I guess, our experience is that a lot of developers, really good developers — even though I wasn’t working in networking for a very, very long time — have a very nebulous definition of what REST is. So if you ask 5 different developers, you might get a very different definition of what REST is and what makes the service RESTful. And that’s partly just because REST isn’t so much of a standard; it’s really a set of principles for architecting a backend. So it’s difficult to say, “Falcor is like REST and it’s like RPC” because, in fact, people don’t really have those touch stones.

So usually when I talk to UI developers, I talk to them about MVC because that’s something that UI developers understand. That’s something that they work with everyday. And then I work backwards to talk about network. So that’s okay. I’d like to actually approach it from that direction.

CHUCK: Sounds good to me!

WARD: Yeah. We can avoid the whole REST fit fight, which [inaudible] go on.

JAFAR: [Laughs] Right. But I mentioned what Netflix wanted. We had the mostly static catalog and we were doing something like MVC, and there’s an interesting MVC. Even MVC is something that can be considered somewhat controversial. But I think, if you look at a lot of Angular apps, the way they’re built, you have a Controller that’s responsible for marshalling information, marshallling the Model effectively for the View and then handing that Model off to the View. I think a lot of people, including guys over there reacting, have been asking about whether MVC really does separate concerns. When I say separate concerns, what I mean is: can we really change the Model, and the View, and the Controller independent of each other because that was really what it imply by separating concerns. I think in practice in a lot of the Angular apps that you see out there, that’s not really true. For example, let’s say I want to add an extra field to a View, like you’ve got a House of Cards and you want to add an extra field that includes the people starring in House of Cards, at least the way it was working at Netflix, and I think it’s true with a lot of apps out there because the Controller was responsible for retrieving that information, we had to go and change the Controller or change the service that the Controller was calling, which is still sort of a Controller repsonsiblity, to make sure that we retrieve just that extra field. In effect, that Controller in their View kind of have two different incentives, really. The View wants total freedom. The View, if you’re just focusing on the View, you like to be able to display anything. And in the entire Netflix graph, you like to be able to display a list of related titles maybe for House of Cards and then you can end up navigating many, many different relationships in the Netflix graph in one View. But the Controller’s incentive, understandably, is to retrieve just enough information for the View because data access is so expensive.

Those sort of two different motivations are what leads to this implicit coupling between Controller and View. So because the Controller is handling the data access about the job of going on the network and getting the data and the View is handling displaying it, well, you have to change both when you want to add an extra field if you want to make sure that you’re efficient. Does that makes sense? That issue of coupling with MVC?

WARD: Well, I hear you…

JAFAR: Maybe I’m a little quick there.

WARD: No, no. Starting with your definition of Separation of Concerns…

JAFAR: Oh!

WARD: But, I don’t want that to interfere with where you’re going so carry on!

JAFAR: Well, put simply, I’d like to be able to change my View without touching my Controller. That’s what I define as a clean Separation of Concerns in this particular case. And if you think about it, well, let’s imagine for a second that there was no network and I had all data in memory, like I literally had a big JavaScript graph of all the Netflix titles of everything. Would I be able to achieve that goal? Absolutely, right? The Controller we just picked a root node out of that big JavaScript graph — just JavaScript objects with references to other JavaScript objects — and it wouldn’t have to decide how much data to go get; it would just pick the right root node for the View like a title, for example. A movie detail page would accept a reference to a title and then the View would just sustain whatever information it wants so the Controller would be responsible for picking the Root Node, the Model, to hand the View. But then the View would actually traverse as much or as little of that graph as it wanted to. So that’s what I mean by Separation of Concerns.

WARD: Yeah. And I think that’s often the way people do it, like the Controller, by way of some service grabs, let’s say a person. And if a person is a complex object, you know graph that make him wander down the relationships the View can while the Controller has done its job by delivering the person object. Or, in your case, the movie object, they could just wander the graph from one movie to wherever and as far as the Navigation pass on that object graph allow. Is that kind of what you mean?

JAFAR: That’s kind of what I mean, but I take exception of the way that’s the way people do it. Here’s the problem: I think when we think about when we build Angular apps or MVC apps, we think about things in terms of Models, like I have the Model for this View and I have this other Model for that View. That implicitly, as soon as you talk about multiple models like I have this Customer Model and I have this Order Model or I have this Movie Model, you’re implicitly describing a boundary for that Model. But here’s how we at Netflix think about our data out there in the cloud; we think of it as one big Model. We think of it as that doesn’t mean it doesn’t have movies inside of it and an individual things that we can call movies, but the reality is it’s a big graph of inter-relationships. So if you’ll ask me to download a video, well, what do you mean? Do you mean House of Cards, the information about House of Cards, like the description and the title of House of Cards and the rating? Or, do you mean the list of seasons and the list of episodes? Okay, maybe that’s what we consider to be the video, but… Wait a second, House of Cards actually has a set of related videos. Maybe that’s what we need to download. That would be a legitimate thing to download if the View had to display a list of related titles — you can easily envision a UI that did that. But then, if I’m downloading the related titles and those are videos as well, well, they’re related to videos and they’re related to videos, and because almost every interesting domain models are graph, unless you pick some sort of boundary — the Controller picks the boundary because it’s the one doing data access — downloading a single video from the Netflix database and its entirety could mean downloading the entire graph. Does that makes sense, what I mean, when I say “the Controller does make a decision — it has to pick a boundary about what really constitutes a model, but the border of what a video is is actually a good deal more for us than one might think”.

WARD: I think I see what you’re saying. I usually make the separation between Model — and I understand Model as you do as it’s about your graph — and the Controller’s responsibility, which I think of as acquiring the root of whatever it is that the View wants, the RootNode, and whether there’s a possibilty of traversing that RootNode to get to that possible related information however you do it. What you’re saying is that traditionally, that would have been the Controller’s responsibility, and now what you want to do is encapsulate that responsibilty inside the node itself somewhere else so that the View can call upon the node and ask it for can start walking those paths? Is that the idea?

JAFAR: I think that’s a good way of describing it. I would describe it in such a way that it basically behaves exactly the same way as if there was no network at all.

WARD: Right.

JAFAR: So instead of the Controller saying, “I’m going to go get these 17 properties for a video,” it would just say, “Look, I’m going to give the reference to the video to the View,” and the View is then going to go and ask for whatever data it needs. And in the process — now that all the data was in memory, that would be a bunch of hash lookups. But if the data isn’t in memory, if somehow the Model is, in our case — and this is really the key — if the model is asynchronous, then all of the sudden, the View can say, “Hey look, Model, get me these 15 properties,” and the Model has the flexibility to do that right when the View asks for it. In other words, we delay data access until the View asks for data. At which point, we know exactly what the View is going to ask for, and then the Model can send the request to the server to get precisely what the View asks for.

WARD: From the Views perspective, it doesn’t know whether that Model object needs to go anywhere, or whether it already knows, that’s the responsibility of the Model — to go figure out how to get it if it doesn’t have it.

JAFAR: Exactly. The only thing we’ve changed — and you think about it — MVC that has 3 parts. It has Model, View, and Controller. And the communication between M, V, and C, the communication is asynchronous, has always been aynchronous with the exception of one particular link. For example, when the View talks to Controller, usually it’s because you’ve clicked an event, right? You clicked the button and then the View goes talk to Controller, that’s asynchronous. When the controller goes and retrieves the Model information  often act as over the wire, well, that’s asynchronous. But for some reason, a lot of MVC frameworks will force communication between the View and the Model to be sychrounous. In other words, when you hand your JSON object to a template, it’s expected that the JSON Objects are already there and synchronously accessible in memory. And that odd a symmetry of saying — well, for some reason, View and Model communications always got to be synchronous — is actually what leads to the tight coupling between View and Controller. If you take that last link, and you say, “You know what? That can be asynchronous as well,” we can actually get full separation of concerns becasue now, when the View says to the Model “give me these 15 properties”, the model can go to the database at the last minute and grab those 15 properties and deliver them asynchronously.

WARD: So would it be fair to say that when a View binds to a property on a ModelObject, every such binding is effectively an asynchronous connection that could return immediately perhaps if I was there or it might take time?

JAFAR: I will say I’m actually going to be very careful about saying “asynchronous”. What I’m going to say is “PUSH” instead. What I mean by that when I differentiate between asynchronous and PUSH is that, let’s say you hand me a callback and now this is the in nature of Reactive Programming, which is that “I call you, you don’t call me; you hand me a callback”. What you’re trusting is that at some later point, I’m going to call your callback.

But there’s nothing necessarily implied by you handing me a callback that says that that process has to be asynchonous. If you hand me a callback, I might turn around and call that callback immediately. So that would still be synchronous. But the difference is, instead of pulling — and what I mean when I say pulling is when you get the return value of a function in the return position. So if I go 2+2, I’m getting the result to the left of 2+2. If I go -x=2+2, that result is kind of coming out to the left where blocking until it’s done and when we’re turning a result — that’s pull. That’s Synchronous Programming in JavaScript.

But there’s also PUSH, where I hand like we might have seen a node, for those of us who use node, or you hand a callback as the last parameter to a function and then that function holds the callback and sense the result of the function in the argument position of a callback. That’s what I defined as PUSH. You’re not receiving data in the return position of a function; you’re receiving data in the argument position of a callback. That means promises, that means events, and that means observable as well, which is the type that Falcor uses and Angular 2 has a lot of support for and its events remodeled as.

So to come back to your original question, the answer is yes, data is pushed to you not necessarily asynchronously, but it might be asynchronous. The wonderful thing about PUSH means that if you get me a callback and the Model has got that data in its cache because you’ve requested it earlier, it can immediately fire that callback and send that data. And if it’s not in the cache, it can make a request and then just fire the callback when data is available.

WARD: That makes total sense. I see where the distinction you’re making. It would be structured so that it could potentially be asynchronous, but that could just flip right around and give you the answer right back now without to actually having to do anything. So the structure, if that’s what you’re describing there when you say I might binding this effectively I callback and it’s the Model’s responsibility to make that callback and to pass that value in when it has it, which might be right now or might be after some time.

JAFAR: Right. When we’re talking about Falcor, we usually say an Async Model to simplify — besides basically what Falcor is offering — it’s like a model for accessing plain old JSON data. The difference is the APIs are asynchronous so we provide the 3 simple JavaSript operations that you’re used to dealing with the most JSON objects: Get, Set, and Call. Get and Set were just for retrieving properties. The only difference is, instead of giving you the data back synchronously, it will give you the data back asynchronously. That gives us the flexibility to let the model do the data access.

CHUCK: I can see somebody kind of go on… Performance.

JAFAR: Right, Performance! Performance is definitely important. And if you mean Performance in terms of the cost of a PUSH versus the cost of reading data synchronously, well, PUSH is definitely more expensive, but it doesn’t have to be tremendously more expensive. If you hit me a callback and I immediately invoke that callback synchronously, yeah, that’s more expensive than returning that data to you. But it’s not like we waited a turn and then get a browser or render and a reflow. So it really comes down to, in terms of measuring performance on invidual platforms, it’s just going to be up to the particulr problem you have. I mean, if you’re trying to observe a large list, well then, maybe the Reactive approach isn’t for you. I want to emphasize that what Falcor can always do is you can basically say to Falcor, “Hey look, grab me as much of the graph, grab me these 17 properties of these 17 objects from a graph and give it back to me as JSON”. And it’ll hand you a JSON object and you can have that JSON object if you want. You can effectively ask it for data and just use it effectively as a data loading mechanism, ask it for data and then just take JSON and bind it directiy to your template, if that’s what you want to do.

But there are a lot of advantages to leaving the data inside the Falcor. Because imagine things like — as soon as you take data out of Falcor, usually what happens is you turn it into a tree because JSON is a tree. And one of the things about most domain models as I mentioned earlier is that they’re graphs. That every interesting application Domain Model is a graph. And the problem with using JSON to send information, although it’s ubiquitous and easily understood by JavaScript clients, is that most application service have to sort of lean on some sort of unpleasant strategy for turning that fragments of those graphs into trees. The most common one is the Duplicate and Identify Strategy, which is to say that if I have two movie lists and they both have the same title in it, in order to turn that into JSON, I’m actually going to make a copy of that title and I’m going to put an ID on each ones of those titles and I’m going to send it over to the client, and presumably the client, I hope, the client when it caches that information locally, is smart enough to dedupe those two objects by identifying the ID.

JSON creates a real hazard because it copies objects. And if I have two copies of House of Cards in two different movie list, a new rate 1 to be 5 and then you scroll down and you see that your rating hasn’t taken into effect, that’s a serious problem in a lot of applications.

WARD: Yeah, you have to have object identity, you have to have some kind of motion of an identity map. In order to make sure that that happens and you’re right, your serialization over the wire should, in order to be compact, deduped before it goes over the wire.

JAFAR: Yes, very few people do that. Or, they rely on usually handcrafted caches. Like if you want to build a cache for a single page web application, you’re rarely leaning on HTTP caching because it’s too course grained. If I make a sinlge request to the server usually in web applications, you’re downloading a lot of different individual items which could change at different times rather than making a single HTTP request for one resource like we did, say, in the World Wide Web days. And this is where we get in to the conversation about REST because a lot of people tell you, “Hey, why are you caching inside of your application? Just rely on the HTTP cache, follow RESTful principles, and you can use the [incomprehensible] cache to do your caching for you.” Well, I think there’s a lot of talk about REST and there’s some really good ideas in REST, and REST is not really practical for a tremendous number of web applications. One thing that’s a little frustrating for me is that there’s a lot of talk about “build your web services as REST” when I don’t think, my personal opinion, is that’s not really appropriate or practical architecture for the backend for most web applications out there. And the reason I say that is that REST came out of a time where — really, it came out of the World Wide Web; a set of architectural principles derived from what worked really well the World Wide Web, which is a lot like Netflix. It’s a big set of resources that you can browse. A big set of mostly static resources that you can browse. So if there’s one service out there that makes a lot of sense to model as RESTful, you might say it was Netflix because here we have this big catalog, data that doesn’t really change very much in the context of a single user session. I mean, why can’t we just rely on browser caches? Why can’t we just make every single one of our videos a URL, effectively a resource when you go there and get the video information.

Well, the reason why we can’t do that is we can’t make 70 concurrent network requests you load up Netflix and we want to display you 70 boxshots. If you take RESTful principles here to their logical conclusion, that’s really where you end up with the Nextflix API, and it’s just not practical. What happened was, we used to use REST on the World Wide Web when we loaded a web page, we’d have a small number of very large resources. You’d have a big document with a couple of big images. An HTTP is really well optimized for doing that — for retrieving large horse-rained resources. But nowadays, when we deal with the web applications, a single page web application, they work with large numbers of very small resources. When I’m using the Netfilx app, if I just swipe my hand on the iPad, I might view a tiny little piece, granted, but I might view 200 resources fly by in the space of the second. So making a single HTTP call for each one of those videos is just not practical.

WARD: Yeah. It might be that each of those little resources has its own resource address. But no one can imagine that there’s also another resource address that is actually the aggragate of those or that you’ve post it, whose response is that collection. And that happens all the time on the web, right?

JAFAR: Well, it does happen all the time, yes. But the question, of course, is: Is that really RESTful at that point? And this is where people get into arguments. But I think, if you take a look at the…

WARD: Ah ah ah… Let me push back on that.

JAFAR: Sorry.

WARD: If I say give me all the companies that identifies a single resource from a REST perspective, it’s called “SomethingSomething/Customers”. Now, the return payload of that contain hundreds of customers each of which is its own resource and would have a typical the RESTing kind of way, it would have its own resource identifier, which would be like “BlahBlahBlah/Customers/BlahBlahBlah/Customers/2”, etcetera. So you are getting a payload that contains objects which are independent resources when in reference of all independent link, but I don’t actually asked for them that way. I asked for them by all customers because I don’t want to go after the eeny meeny miny moe, right?

JAFAR: Right. And that’s why I think we’re bending the rules to some degree. And I think almost everyone does. I want to be clear about that. I think everyone bends the rule. And the most particular case, the reason why we’re bending the rule is that I think if you really are applying RESTful principles, you don’t get back that customer object; you get back a link to that customer object. And there’s a very good reason why in a RESTful system, you get back a hyperlink to the customer object and not the customer object itself. It’s that, let’s say I turn around and I request that customer object the next time around using its URL, now I have two places in my cache, two resources that overlap the same customer object. And that’s a serious problem. If I want to do [incomprehensible] an operation like completely replace that customer object, I know I need to do that on two different resources in order to make sure I don’t have stale data in my cache.

So my point here is if you want to, you can sort of put the same object in many, many different resources but then you have to be willing to deal with the reality, which is that you make it stale caches then if you rely on your browser. And you and Netflix, I can tell you, were not okay with somebody rating a video and then scrolling down and not seeing that rating represented. It’s not good usability. I don’t think that’s unusual. I think a lot of web applications out there would not be okay with that. So if you essentially bend the rules on REST and say, “Yeah, you can get this aggragate resource,” which actually contain these 50 another resources, which I contend is not RESTful, then your effect is they’re going to run into this and stay on this problem.

CHUCK: Yeah, but then how do you deal with the issue of, okay, I’ve got the list of, say, all companies are all movies and I have the screen set up so that it shows me 15 or 20 movies. I have to make 15 or 20 requests to get that information.

JAFAR: You don’t! [Laughs] What I’m saying here is: It doesn’t work both ways. Either you deal with stale data or you deal with terrible latency. And in the mobile world, it just doesn’t work. It’s not really a secret. Roy Fielding said himself that REST was designed for large grain resource — I hope I’m not misquoting him here, but large grain resource kind of transfer. And as soon as you got a web application, you’ve got a lot of small web resources, where a resource to you could be a JSON object smaller than the HTTP headers of maybe a typical HTTP request, you got to look at the signal, the noise ratio, and sort of say, “Is REST solving my problem?” And my answere is, it’s definitely not solving my problem, and I don’t think it’s solving the problem the most web applications out there. But I think that nobody seems to be, at least in my experience that I found, nobody seems to be talking loudly about this problem. Although, I think actually you’re seeing other folks, like the React folks and Graph QL, I think they’re coming out to try and address this problem. But I think that sort of where I am in terms of using a really RESTful API to work for Netflix — I don’t think it’s going to work.

But that said, REST has got some great ideas. This whole idea of idempotent resources and idempotent operations on those resourcess and the ability to cache things by their unique identifier, all that staff makes tremendous sense for Netflix. Every single one of our little video resources could be thought of as a thing that we’d like to cache somewhere, almost never changes, and we’re not really dealing with hypertext, we’re pretty much dealing with JSON. We’ve got structured data where REST is kind of described in a way that’s very hypertext-centric. What we wanted to do is we wanted to take the ideas behind REST. Some really good ideas. And we wanted them to work for web applications and work at the granularity of resources that web applications deal with. And really that was our starting point for Falcor. Our starting point was, “Well, let’s see how the system changes,” and this is really at the network level. The one constraint that we apply is we say, “Let’s see how the tradeoff change if we say this application gets all of its data from a single domain, all or most of its data from a single domain”. REST was designed as a distributed architecture where I mean you could make a mash up where you get your data from 16 different domains, and that’s great. That’s wonderful and that’s flexible. But a lot of web applications out there, you know what, they go to one domain they get their data. They’re not mash ups, they’re going to one particular domain to grab that data and modify their data. And if you apply that additional constraint, then some of the things that REST does where I download a list of customer URLs and then I follow that list of customer URLs back to the exact same domain, well, that doesn’t really make a lot of sense. Definitely, it makes sure that each individual HTTP resource doesn’t overlap within the other HTTP resource. But it seems a little silly to request the resource from a server which then contains 10 more links to go back to the exact same server. So if you fall in to this category of applications that I know there’s a lot of folks out there who do, where your application is getting all your data from one domain, well, why not still maintain this idea from REST, which is that you have resources which don’t overlap with other resources. But, you can follow links to those resources on the server instead of sending them back to the client and having the client follow them, which you would need to do if you got your data from a bunch of different domains. Once we set the constraint that we’re getting our data from one domain, why don’t we just follow those links on the server beforehand and then get all the data we need as one big block back to the client instead of forcing it to keep jumping back and forth to the server, which means tremendous latency as a non-starter mobile.

WARD: I certainly see the problem that you’re describing. And I think, for example, OData (just picking one) is in with its expand so that you can build, you can make a grid against graphs and staff like that. I’m kind of reflexed the concern that you’re describing there as well. Do you see the correspondence there? Or, did I just introduce a technology that wasn’t for me to do?

JAFAR: No, no! I know OData. I used to work at Microsoft and I know their concept of expand although on a little bit of been 6 or 7 months since I took a look at their data. I guess my concern around at the time when I looked at their data and — well, I want to be cautious because it’s been a while since I’ve taken a look at it — it didn’t quite have the level of composition that I was looking for. And in terms of being able to go infinitely deep into the graph in a single network request, but maybe that has changed, it’s been a while since I’ve taken a look at it.

WARD: Well, you know, details aside, it’s kind of got that same concern about like you want to say for a particular grab. I want whatever it is my filter is on a node and then I want to go down only so far in a particular request down the path of a latent objects. And I’m guessing that in Falcor, you do something like that around the movie where you, in a single sip, in a single request, you’re going to get back a portion of the object graph that is sort of extend so far, but not infinitely otherwise you’d pull the entire database down. Is that approximately correct?

JAFAR: That’s precisely it.

WARD: What I found really interesting when I think I saw one of your earlier Falcor presentations is, let’s suppose I got a certain part of the graph and now I’m binding to it into UI, as you say, it’s sort of its designed as if anything I could request in the UI in the View might not be there, right? So it’s all kind of an asynchronous structure. So if I go for the movie title, that might or might not be there, right?

JAFAR: Right.

WARD: But, most of the time it is so therefore it turns right around and puts the title up and everything’s great. But in any point, the View might ask for something that really isn’t there, and then Falcor will transparently Lazy Load it. Is that of your statement?

JAFAR: That’s correct.

WARD: See, I think that’s cool. I think that’s really interesting and that’s only possible if you structure your View binding such that anything could potentially either be there or be Lazy Load it. Is that fair to say?

JAFAR: Yes.

CHUCK: So basically, what I’m hearing is that when you make the request, do you sort of warm up the cache so to speak? In other words, you get the data that you’re likely to need in one big chunk? And then anything that’s not there, then you go make the odd request to get that stuff?

JAFAR: Yes. I mean, you can delay all data requests until really like that View displayed. I want to call out, you know, Falcor — there’s no silver bullet to data access [chuckles]. Data access is a phenomenally hard problem and part of the reason is that you’ve got fundamentally — and it comes up, in Computer Science, again and again — this tension between the desire to be abstract in order to have less knowledge about what other components are doing and the overwriting need to be fast. So I don’t want to know what component he is doing, but gee, if I only knew what component he is going to do, I might be able to do things much, much more efficiently. So I think the industry is grappling with this problem with User Interfaces and data access, and it’s very tough to try and find the right balance. I think, I want to definitely be the guy who calls out some of the challenges with this architecture, which is why now we’re calling Async MVC, but that basically just means you’ve got a model with the PUSH interface and you request your data at the very end. You can also end up with request patterns that are not as optimal. If you wait until you display data in order to go get it, you can end up with a REST pattern that’s not very optimal because you would end up in like sequential request. Let’s say I display a Customer Object, and then that customer contains a list of words, and then I make one with Falcor. What it’s going to do is it’s going to take all concurrent request and it’s going to batch them. So if you ask Falcor for 15 different properties in the same turn, in the same turn of the event loop, it’ll make one request for all of those. But that’s not a silver bullet because now, let’s say, we get back to list of orders or at least the list of the ideas of the orders and then we try and display, and then we find out, “Oh, you know what, we don’t have the name and order number of these things.” And then we might now need to make a sequential request for the name and order number of the first 5 orders. Now, those are all get batched up in one request, but that’s still two sequential request for what you could have made with one request, if you are willing to just repeat yourselft basically. If you’re willing to say, “Hey, I know the next View is going to go and grab these extra properties so I’m just going to have Falcor go ahead and preload all these things so by the time it comes up when, although we’re still going to sort of request to the Model for the data, it’s just going to come out as cache”. Does that makes sense? Async MVC can defniitely give you a full decoupling, but full decoupling can sometimes lead to some optimal network request patterns.

WARD: It’s a really difficult problem to decide. What you’re going to load and what you’re going to Lazy Load. I’m totally with you there and it needs a little push, a little hint from the developer usually, unless Falcor is really smart and has recognized previous usage patterns and automatically adjusts itself. Now, that would be super cool [laughs].

JAFAR: It’s interesting to think about what that would look like. One approach would be yes, some sort of statistical kind of, “ah, well,” you know. Falcor as a client, probably can do that in a sense that it’s stateless, right, but maybe if the UI like some plugin to Falcor could look at patterns of sequential requests to somehow infer them, I can tell you, that’s definitely not on the road, not for the open source. But I’ll tell you this, what I’m really excited about, I’m excited about with Angular 2 because Angular 2 has got a lot of great features.

Here at Netflix, we use React for our public facing UIs, and we like React. But that doesn’t necessarily mean that we definitely want state of MVC, as of everybody, I think, the state of MVC to get better. And there’s a lot of things that the Angular guys are doing that I’m really excited about. One of them, as I said this notion of allowing observables, which is the data type that Falcor emits to be bound to declaratively in template. The unfortunate things about having a PUSH Model, this callback-based model is, it can become the cumbersome obviously the code against. And that’s easy to just pull a property out of the model. But if you’re taking your modesl like most, I think, Angular developers and you plugin them directily into Views, Angular, it looks like it’s going to its extensibility mechanisms way of just declaratively binding to asynchronous information, which is a feature that I think absolutely ever your MVC framework should have. In the JavaScript space, there’s not really a very good reason for having template languages that don’t know how to bind especially in a world of promises in JavaScript, don’t know how to bind asynchronous data sources. That’s the decision I think they’re making that I really, really think is the right decision.

So using Angular and Falcor together should be quite easy because you can just pretty much write the same code you have written had you been binding in the memory JSON object. You only need to put a little bit of ceremony in the template to bind you an asynchronous object. And I think as time goes on that amount of ceremonies is going to go down anyways. That’s one thing I’m excited about.

But what I’m really excited about is the potential for each of these technologies to solve each other’s problems. So you were talking earlier about knowing what to Lazy Load and knowing what to Eagerly Load. That’s definitely not something that I don’t think there’s one solution to that problem because if you think about it, the idea might be based on the runtime, like at runtime, things like latency and bandwidth, right? There’s no one size fits all solution to the right data access patterns, but there’s some pretty good start, there’s some pretty good guesses we can make.

So I want to come back to what I think Angular and Falcor can do for each other that’s really exciting. I talked about the Angular MVC issue, and I think it is an issue. This coupling that you get from Model, View, and Control — it’s not just an issue with Angular, I don’t mean to pain it; it’s just an issue with Angular. I think you see it in a lot of different MVC frameworks. This implicit coupling between View and Controller. Well, this Async MVC pattern, the people used at Falcor also has an issue, which is what I talked about earlier about when you wait to retrieve information to write the moment before it’s displalyed, you can end up with sequential data access patterns where you might have gotten the way with just one big day load ahead of time that you knew what was going to be displayed. I think Angular and Falcor, when used together, can actually get as closed as we can get to the global maximum for data access patterns and  decoupling. So we can have it for a cake and we can eat it, too, which is a bold statment. We can have separation of concerns where I can change the View, and bing! It just shows up when I reload the app; I don’t need to change my controller or anything.

So we’re going to have separation of concerns and we can also have really efficient data access patterns where we can make one request to the server to load all of the data dependencies for that view.

LUKAS: It might drop.

CHUCK: Yup.

JAFAR: Right.

CHUCK: I hate to stop you, but…

JAFAR: Oh, you’re kidding me.

CHUCK: We’re running out of time.

JAFAR: [Laughs]

LUKAS: Part 2, anyone? Yes? Yes?

CHUCK: Do you want to just come down with the whammy and then we’ll come back around and do another episode?

JAFAR: Yeah, sure! By analyzing Angular template at compile time, we can build up a list of the properties that that template is going to need — the entire template, and its child templates — and then when the Falcor is loaded when you bind the Model to hand to View, we can use that extra information to give Falcor the data it needs in advance so we can make one request to the server. So you can use the template information to inform the data access. That’s how you balance both decoupling and performance.

CHUCK: Awesome.

LUKAS: That is amazing!

CHUCK: Yup.

LUKAS: We need a part 2.

CHUCK: Yeah, we do. Let’s go ahead and do picks and then we’ll schedule another round.

JAFAR: Sounds good!

CHUCK: Lukas, do you have some picks for us?

LUKAS: I have 2 picks. The first one is, thanks to Jafar’s insightful prodding, I am digging into lisp and so the book that I am going through right now is “Land of Lisp” by Conrad Barski and it’s really quite awesome.

Now, for the pick of picks, I would like to pick a song called “1-2-3 Sesame Street” by Stevie Wonder and it’s pretty much just the greatest song ever.

CHUCk: Awesome.

WARD: [Laughs] I can think of a lot of Stevie Wonder songs. That’s the one you picked?

JAFAR: [Laughs]

LUKAS: I’m telling you, like he’s autotalk box and it’s just ridiculous.

CHUCK: Ward, do you have some picks for us?

WARD: I Just Called To Say I Love You.

JAFAR: [Laughs]

LUKAS: From the bottom…

WARD: Oh, okay. Well, Lukas just killed me there. I’m just completely lost.

JAFAR: [Laughs]

WARD: But I think what I want to talk about is one of the many, many, many things I saw at build last week that I like was “Visual Studio Code”, which is a cross-platform somewhere between and IDE that I sort of attended that hit the sweet spot for what developers do all day long. And I’ve been playing with it. For V1 Preview, it’s very impressive. It might very well become my go-to tool of choice for developing both HTML JavaScript Apps and to some degrees some C#, perhaps some TypeScript apps. So, worth taking a look at.

CHUCK: Alright. I’lm going to throw out some picks. The first one is a book called “Traction”. It’s by Gino Wickman. I might have picked another book called Traction before, don’t get confused, there are two of them. This one is the one that doesn’t have the yellow cover. This one basically walks you through how to organize a successful company. And since I kind of run a company that does things with the podcasts among other things, I thought I should read through it. I really enjoyed it. I’m rearranging some things with my business. So yeah, if you have a small business, this is definitely something worth reading.

I also just finished “Wool” on the bus edition. I think Lukas picked it on the show before. Super good. So if you’re looking for kind of fiction book, I’ll go ahead and pick that.

And then I set up a “MediaWiki” site. That’s basically a few views Wikipedia, it’s the Wiki that runs Wikipedia. The reason I did that is because as part of this organization thing, I want to start documenting the procedures that I have in my business and I wanted them available online for the people who are doing work for me so I set that up. It wasn’t horrible to set up. There were few things that I had to kind of Google to figure out, but I figured them out pretty fast and so far, I’m liking it. So I’m going to pick MediaWiki as well.

Jafar, do you have some picks for us?

JAFAR: One thing I’m really playing a lot with is a “PureScript”. I don’t know if anybody out there heard of it, but it’s a little JavaScript language, one of the compilers of the JavaScript languages. It allows you to write code like high school, which compiles in the JavaScript, although, I haven’t actually use it on project yet. It’s actually a very different and interesting way of approaching web developing.

CHUCK: Let’s go and wrap up the show. Thank you all for coming. And we’ll catch you all next week!

[This episode is sponsored by Mad Glory. You’ve been building software for a long time and sometimes it gets a little overwhelming; work piles up, hiring sucks, and it’s hard to get projects out the door. Check out Mad Glory. They are a small shop with experience shipping big products. They’re smart, dedicated, will augment your team, and work as hard as you do. Find them online at madglory.com or on Twitter at @madglory.]

[Hosting and bandwidth provided by The Blue Box Group. Check them out at bluebox.net]

[Bandwidth for this segment is provided by Cache Fly, the world’s fastest CDN. Deliver your content fast with Cache Fly. Visit cachefly.com to learn more.]

[Do you wanna have conversations with the Adventures in Angular crew and their guests? Do you wanna support the show? Now you can. Go to adventuresinangular.com/forum and sign up today!]

x