025 iPhreaks Show – The Accelerate Framework with Jeff Biggus

00:00
Download MP3

Panel Jeff Biggus (twitter blog) Rod Schmidt (twitter github infiniteNIL) Charles Max Wood (twitter github Teach Me To Code Rails Ramp Up) Discussion 00:32 - Jeff Biggus Introduction HyperJeff Network 01:38 - The Accelerate Framework Audio Video Images Pure Math Functions 11:11 - Accelerate Framework Tricks 12:35 - Clang Extensions 17:30 - Libraries 23:06 - Gotchas 28:39 - Mac -> iOS 31:25 - The Convolution Integral 33:49 - Finding the Accelerate Framework 35:35 - Audio Picks F-Script (Rod & Jeff) Nathen Harvey: Learning Chef (Chuck) Start your free trial of Enterprise Chef (Chuck) Chef (Chuck) Cocoa Literature (Jeff) Next Week Custom UI Controls with Sam Davies Transcript Transcript CHUCK: Hey everybody and welcome to Episode 25 of the iPhreaks Show! This week on our panel, we have Rod Schmidt. ROD: Hello from Salt Lake! CHUCK: I'm Charles Max Wood from DevChat.tv. I just want to give you another quick reminder that you can find out about my freelancing journey at GoingRogueVideo.com, it's a 30-minute video detailing how I went from laid off to freelancing successfully. We also have a special guest, and that is Jeff Biggus. JEFF: Hello! CHUCK: Since you haven't been on the show before, Jeff, do you want to introduce yourself? JEFF: Sure! I'm a Mac and iOS programmer, I work for my own company, have for about 17 years now. Lately, it's been all Mac and iOS, maybe for the last 7 years or so, and I come from a background that's maybe more science, like academic science, but now I do a lot of just general programming stuff. CHUCK: Awesome. What kinds of things have you been academic-ing or studying or whatever? JEFF: [Chuckles] I used to do a bunch of computational physics stuff. So I was in the physics world for a while, and before that, I was an electrical engineer.  It was a lot of very practical engineering type programming. Ugly code that got the job done. CHUCK: Boy! That sounds like what I do! JEFF: [Chuckles] CHUCK: Ugly code that gets the job done. JEFF: Exactly, yeah. Which is basically what I do now; except it's a little more elegant these days. The tools are much nicer. CHUCK: Awesome. So, we brought you on today to talk about the Accelerate Framework. JEFF: Yeah. CHUCK: It's part of the standard suite of the libraries you get from Apple, it looks like. Can you explain briefly what it is or what it does? JEFF: Sure! It's a set of, really, C Libraries. It's called the Framework. Normally, when you hear framework, you'd think Objective-C framework, but this is all C-level calls. The goal of it is to give us set of functions, a humongous set of functions, that can allow you to do common tasks much faster because they're heavily optimized. As a consequence, that actually saves battery life so you get more power and better energy use. I would even like the case that if you get into it, that your code is actually often better; it's cleaner and it's more maintainable going forward. CHUCK: Huh! What exactly is it used for? JEFF: When you look at the framework, it looks like there are basically 3 areas that it seems to hit which is Audio, Video, and Images, and also Pure Math functions. But, since everything under the hood is all just bits, sometimes, whether it's something is an image or audio or math, it's just that your perspective on things. A lot of times, these functions that are in there can be used for even really mundane tasks that you might not be thinking about them as. You might grab an audio function to do something where you’re simply doing some very simple calculation for something that has nothing to do with audio. CHUCK: Got you! ROD: I would imagine game programming would use a lot especially 3D… JEFF: Oh, sure! Game programming seems like the easiest place to get the most immediate impact from this, but also, Video and Image Processing. Actually,

Transcript

CHUCK: Hey everybody and welcome to Episode 25 of the iPhreaks Show! This week on our panel, we have Rod Schmidt. ROD: Hello from Salt Lake! CHUCK: I’m Charles Max Wood from DevChat.tv. I just want to give you another quick reminder that you can find out about my freelancing journey at GoingRogueVideo.com, it’s a 30-minute video detailing how I went from laid off to freelancing successfully. We also have a special guest, and that is Jeff Biggus. JEFF: Hello! CHUCK: Since you haven’t been on the show before, Jeff, do you want to introduce yourself? JEFF: Sure! I’m a Mac and iOS programmer, I work for my own company for about 17 years now. Lately, it’s been all Mac and iOS maybe for the last 7 years or so, and coming from a background that’s maybe more science, like academic science, but now I do a lot of just general programming stuff. CHUCK: Awesome. What kinds of things have you been academic-ing or studying or whatever? JEFF: [Chuckles] I used to do a bunch of computational physics stuff so I was in the physics world for a while. Before that, I was an electrical engineer.  It was a lot of very practical engineering type programming, ugly code that got the job done. CHUCK: Boy! That sounds like what I do – JEFF: [Chuckles] CHUCK: Ugly code that gets the job done. JEFF: Exactly, yeah. Which is basically what I do now; it’s a little more elegant these days, the tools are much nicer. CHUCK: Awesome. We brought you on today to talk about the “Accelerate Framework”. JEFF: Yeah! CHUCK: It’s part of the standard suite of the libraries you get from Apple, it looks like. Can you explain briefly what it is or what it does? JEFF: Sure! It’s a set of really C Libraries, it’s called the “Framework”. Normally, when you hear framework, you’d think Objective C framework, but this is all C-level calls. The goal of it is to give us set of functions, a humongous set of functions that can allow you to do common tasks much faster because they’re heavily optimized. As a consequence, that actually saves battery life so you get more power and better energy use. I would even like the case that if you get into it, that your code is actually often better; it’s cleaner and it’s more maintainable going forward. CHUCK: Hah! What exactly is it used for? JEFF: When you look at the framework, it looks like there are basically 3 areas that it seems to hit which is Audio, Video, and Images, and also Pure Math functions. But, since everything under the hood is all just bits, sometimes, whether it’s something as an image or audio or math, it’s just that your perspective on things. A lot of times, these functions that are in there can be used for even really mundane tasks that you might not be thinking about them as like you might grab an audio function to do something where you’re simply doing some very simple calculations for something that has nothing to do with audio. CHUCK: Got you! ROD: I would imagine game programming would use a lot especially 3D… JEFF: Oh, sure! Game programming seems like the easiest place to get the most immediate impact from this, but also, Video and Image Processing. Actually, now it got just any medium whatsoever; anything where you’re crunching a lot of numbers at once because you’re trying to achieve a perception effects, something really immediate to the end user, and you want it to be done efficiently and as quickly as possible. CHUCK: Does this kind of take the place of some of the other libraries that you would typically use if you’re not really performance constrained in that way? JEFF: There are times when this is not the best framework to use, but when it is, you will often have to maybe benchmark it against something else. Apple has a library or a framework called “CoreImage”. There are times when CoreImage’s Image Processing is faster than what you’re going to get with one of the Accelerate libraries. There’s one called “vImage Switches” at Vector Image library, and that’s part of the Accelerate.framework. There are times when you have to look at your codes specifically, and try each. Because depending on what size the image is and what kind of effects you’re doing with these images, there are times when one library is going to be better than the other. CHUCK: Is CoreImage easier to use than Accelerate.framework, or about the same, or does it matter? JEFF: I found all of that CoreGraphics stuff difficult, so for me, it’s all about the same. I had bought as easier time; I’m trying to use CoreImage as I did with vImage. I was lucky enough to run into Bill Dudney a few different occasions and he’s a master at this kind of stuff so he was able to negotiate some difficult areas that I don’t know what to do with with CoreGraphics. I’m not maybe the right person to ask as far as how easy it is to go. But there are times, like if you’re doing something like virtual reality type stuff where you need stuff in real time be processed. Often that is the case for the vImage library is going to be optimized for what you’re doing. But if you’re using average sized images, it’s more question of the kind of data that you’re working with, then whether or not the API is easier to use. Although I know on a practical level, how I use the API often is exactly a one-to-one correlation with what people do. Actually, that gets to the point of sort of the Accelerate.framework’s whole problem throughout its whole lifespan. It came about roughly around 2004 or so with a driver on the Mac side, and then it slowly, in iOS 4, became part of the phone and the iPad stuff. But if you go to a session of the WWDC, and you go watch an Accelerate talk or grab the video and watch those, the older video is what really just scare the pants off you when you looked at the APIs… CHUCK: [Laughs] ROD: [Chuckles] JEFF: And everybody who went to those sessions would come out and go, “Oh, my god! It’s not awesome. I’m never ever going to use that.” CHUCK: [Laughs] ROD: [Chuckles] JEFF: So they’ve had this PR problem, and it really is not as bad as it looks. Part of the reason I started giving talks on it at different conferences was if people could see simple examples doing actual things and side-by-side compare it to the code you’d have to write without it, and then also see the giant speed increase and the energy use improvement, it then becomes an easy cell. Because even though it’s a C-level call for a lot of this stuff, a lot of the times, the day that you’re working with comes down to some flaunts and some dollars that you’re multiplying or something like that. And often, the code itself looks better in the end, but you need an on-ramp. That’s something that Apple sort of never really gave in their documentation or example kind of where really simple, here’s how we walk from nothing to do something awesome. ROD: One difference isn’t CoreImage uses the Graphics Processor, and the Accelerate.framework uses the special instructions on the CPU? Is that correct? JEFF: That’s what it does currently. The Accelerate stuff is kind of funny because they sort of reserve the right to do whatever is best under the hood at any given. So while they are using a different set of – I don’t know what algorithm here actually implementing these functions with – but if something is fast around the GPU, they will switch to the GPU. I’m not saying that functions do that now, but as they go, they heavily optimize this code for each CPU that comes out. Obviously, these guys, they’re at Apple. The greatest thing about all this is that the people who are working on this, they’re all with Apple; Apple has got a very well-known set of hardware and chips, and it’s not a zillion different kinds so they can heavily optimize where exactly this chip and they know what the graphics pipelines are. So there are times when a CPU operation can be faster purely because getting the data to the graphics card there and back gobbles up more time than what you say when it finally does something. Also, it’s using stuff like all the extra registers. On the Mac, you have your SSC stuff; and the iOS, you have the NEON registers. So it’s using parts of the CPU that are really generally not being used by general purpose programming. CHUCK: Hah! JEFF: Maybe that’s something worth mentioning. Like, all of the stuff is so optimized that very mundane tasks like if you’re running for a loop, just a super four loop, and you’ve got 2 items that you’re just multiplying, maybe you’re doubling every element in some giant array because you’re just kind of double the size in some part of your dataset. By using these giant registers that are on the CPU, you can maybe grab handful, maybe 16 numbers at a time, do a multiply immediately. And then if the compiler optimizes stuff, maybe you can use the result that’s still sitting on the CPU for the next step of your operation. So, stuff where if you wrote it in a normal way with your general C-programming style where you take a couple of numbers and you multiply them, then you maybe divide them by something else on the next step, it might be going to the CPU taking stuff out of RAM doing the operation, putting the result back in RAM, and then there’s all this traffic back and forth between your cache back to the chip. There are times when these retains in the Accelerate.framework are like, “You know what, the result we need is already stored on the CPU,” we’re not even going to back out, to cache or anything; we’re just going to use it immediately for the next step of the operation. So the speed increase sometimes is absurd. For certain operations, they can go up to a thousand times faster. Now, there are times where it’s best only like maybe 20% faster. That’s why I always encourage everybody to benchmark to use it with their real dataset and to start it play around because sometimes there is a level of playing with your data where you can get 20% increase, but you just play with it a little bit, and all of the sudden, you’ve got 300 times faster. CHUCK: So what’s your favorite trick with the Accelerate.framework? JEFF: My favorite thing, just because it’s such a quick win, is when you’re looking at a piece of code that’s just running in a loop. You look inside the loop, sometimes it’ll send nested loop, and you look at what’s going on in the middle of that loop. You’re maybe taking a sign of something and multiplying it because you’re doing maybe some little effect somewhere. Usually, if you see a loop with math in the middle of the loop, usually you can replace the whole loop with maybe a one or two-liner with no loops, where the loops are all implicitly taking care of by the Accelerate.framework. You just shove it a giant array of data, and you say, “Hey, you know what I want? Every element in this array to do something with some operation,” maybe it’s a filter; maybe you’re doing the masking filter for graphics or something. But a lot of these things, you can turn into one or two-liners in your program. And then all of the sudden, your iOS app that is running the same as everybody else is on the store, all of the sudden, you’re significantly faster at what you do. So those things are sometimes so easy to find and so quick or win. It gets fun in its own dirty way. CHUCK: [Laughs] ROD: I was looking through the Clang Extensions, and it looks like it has some support for the Accelerate.framework or at least using these instructions on the CPU. Do you know anything about that? JEFF: I know that they make use of the features that do become available in like for instance C99 and then C11 in the different specs and then obviously, Apple has moved completely over to client in the LLVM. So they do make you use of whatever they can along the way for these things. That’s all compiler stuff. A lot of the things that they’re doing in the Accelerator.framework, they’ve got assembly level optimizations. So while they do make use of the things, they can make use of incline for sure. They have to make sure that it’s still going to, if Xcode needs to compile back to the same operating system, they got to make sure that it falls back to whatever it is you’re targeting. And there are times, when the CPU you’re targeting, like if you’re still building something for the A6 chip, there’s a lot of stuff the A6 chip can’t do in hardware. So even though you’re using an Accelerate call, it knows you’re going to the A6 chip, so what I’m going to do is it’s going to become a software [inaudible]. It’s going to be hardware optimized like it would be on the A7. ROD: Right. I didn’t look closely at it, but I saw that they even had like a vector type that you could use. JEFF: One thing I should mention about, the other main problem of the Accelerate.framework is it’s really like 5 distinguished libraries, and this is the second part where people say, “Okay, I’ll take a look at this thing because I heard good things about it.” But then you go in there, and you’re like, “There are 5 totally different libraries in here,” and they look completely different. They have different ways of writing functions, and sometimes, they have different data types. Some of them will accept the Cs – C knows, since C99 has a complex number type, it didn’t used to. So the audio library has its own complex number to make life even more fun; it has 2 different types of it. ROD: [Chuckles] CHUCK: [Chuckles] JEFF: It’s a nightmare when you’re first going into there because you’re like, why is the level of pain so high when you started using this stuff. But they do play fairly nice with each other once you get used to the fact that they have different naming conventions. Like the vImage function call is probably the most memorable looking one, and there’s probably the place to start for most people because it’s a C call, but you can understand it, like they named it nicely long and stuff to give it a descriptive image, and it uses standard CoreGraphics data types as input, and it gives you a nice output that you can use. So you can plug that into existing graphics calls that you do now, and you can get your feet where the easiest start. But then the audio library looks a little different; it’s a little older. The audio library goes back to the early 90s because that one started with Next, and those calls got inherited from Next. But then it gets even crazier because the 2 Core Math Libraries are in there, which have ridiculous names because they were made in FORTRAN for people who are used to very [unclear] neat things and have a BLAS library, B-L-A-S, and then an LAPACK library all in upper case, like screaming at you that this came from FORTRAN. And the naming conventions of those functions are 6 letters long [laughs] because it came from the day when you had 6 letters to name your function, so they’re indecipherable; you need like little cheat sheets and stuff to even find your way around in those. And there are times when the documentation on Apple’s part for those libraries are maybe – I think the LAPACK documentation – is coming straight from the Linux load so there’s nothing specifically Mac about; if there’s a Mac issue with any of them, it’s not mentioned in the documentation. It’ll still work just the way you expect, but it’s too old documentation on some of that stuff, and that’s because that stuff started in the 1980s and it’s highly, highly perfectly optimized awesome stuff, but it’s ugly to look at the function things. I end up putting together cheat sheets from the BLAS library just because I found like 4 or 5 of matter, and I couldn’t make heads or tails of any of them, so I just made my own. ROD: Are there any higher level libraries that make use of the Accelerate.framework like maybe OpenCV or something like that? JEFF: OpenCV doesn’t use it; OpenCV has really got something going on, and Apple will often do bake offs against OpenCV and in most areas best it. Sometimes, it’s fairly [unclear] me. Sometimes, it’ll be maybe 50% better than an OpenCV call and something, and sometimes, it’ll be maybe a few hundred times faster. So because OpenCVs are open source and it’s a multi-platform, [unclear] to sort of am that to make use with any calls. ROD: Right. JEFF: There is something you can do. There are times when, if a library is making use of the cross-platform part of the Accelerate.framework which is the BLAS and the LAPACK which are very Math-y kind of things, if you find a set of code out there that makes use of that, those old functions, you can actually just throw a compile flag to say, “Hey, use the Accelerate.framework for this,” and it will instantly make use of the hard or optimized versions of all those functions. ROD: That’s pretty cool. JEFF: That happens pretty rarely because those libraries are generally older or maybe things where Mac programmers are typically looking for libraries. ROD: Are there any other higher level libraries that you could use to make things easier? JEFF: I don’t know of any right now that – here’s the thing about trying to…I had a dream long, long time ago; it was this twisted dream of somehow wrapping the ugliest parts of the Accelerate.framework into Objective C. But if falls apart not only because of the sheer number of functions that are in there; we’re talking like thousands of functions. But part of the reason you can optimize things so well is that when you make a function call, and if it can make some sense of, “Oh, you’re doing these 3 functions in a row,” it can actually do this optimizations that if you try to wrap it in Objective C calls where you’re doing this message passing so instead of having 2 instructions happening one after the other, they’re actually happening maybe 10 steps removed from each other because of all the indirection that happens on a higher level. Then the little tricks that it can pull about leaving the data on the CPU and a certain register so that it goes ultra fast, all of that falls away or at least, again, it’s pretty diminished. ROD: Right. JEFF: But that’s not to say that it doesn’t coming handy if you are using just one or two of those functions. You can wrap your own – I do this all the time. I wrap maybe a little vector class where I do certain operations so that the code parts still looks like beautiful Objective C code. And what’s going on under the hood are these great Accelerate multiplies or something like that. ROD: But, there’s not even any higher level C libraries? JEFF: No, not that I know of. These are pretty high level  there. For instance, there are calls in here. If you’re doing anything involving fast 48 transforms. So, if you’re doing anything that’s analyzing stock market or just looking at photos doing architecture or anything like that, those are really high-level things right there. And there, in this beautiful, as simple as you can get in calls in the Accelerate.framework… ROD: I see. JEFF: So I should say that it’s not super low-level calls that happening in the Accelerate.framework; it’s just that there are C-level. So if you had a lot of data and it’s all in NSArray, then you might want to rewrite so that you have a C right instead because then you can just cram this regular C array into this function, and get the answer out in this other C array. ROD: Right. JEFF: And it’s worth mentioning also that a lot of times, people don’t realize that they can use this framework for very mundane tasks, things that a lot of people will come to the session that I do and they’re thinking, “Okay, the program I use doesn’t need any of this power. I don’t need this be a bruised, I don’t need the energy efficiency, and I’m not doing like Math, Science, or Multimedia. But a lot of times, when you look inside the code, there’s some moment where even that a simple app – I’ll use a correct example…I have an app on the store that tracks book reading habits; it’s just a book tracker. But there’s a moment where I graph something and I wanted when you hit a button, it used to take half a second for the View to slide and would create this graph of certain aspect of that higher reading stuff. I looked in there and I was doing almost like all of the time has been chewed up by me going through the data and just kind of smoothing the data so it’s presentable to user. It turned out there is a function in the audio library, in the vDSP, Digital Signal Processing library – that did it in a one-liner. I added it in there, and now, when you hit the button, the View just slides where it in like nothing happened at all; that comes already ready. CHUCK: Oh, very nice. JEFF: These are really simple iPhone programs [inaudible]. Pardon the truck outside. CHUCK: You were saying that there is actually some resistance using it because it’s complicated or whatever. Are there any gotchas that you run into with using the Accelerate.framework? JEFF: Absolutely [laughs]. There is definitely a learning curve with it. Something was work has gotten taken up the line share of my time this year, but something I had been meaning to did going was a website that would be nothing but simple to complicated examples of using the different parts of the framework. Because even if you’re doing a simple operation, you might not realize that, “Okay, I need – “ maybe you were doing something for whatever reason you need to multiply in matrix times in the vector. You don’t realize that the way you’re giving it the matrix, it’s upside down, like it’s inverted in the wrong order. So their each little function, the best thing to do is to write a very, very simple C program to make sure that it’s doing what you want in the sense that it’s a unit testing yourself to understand the function. A lot of times, the documentation that’s in here, it’s getting better, but it’s probably the least helpful documentation in Xcode – it’s this stuff with the Accelerate.framework. They’d made strides, but it’s still very complicated. So, the answer is, yes, there are a lot of sharp gotchas in there that don’t seem to make any sense. Even when I was trying to get those simple vImage calls, I’m not a great CoreGraphics guy, but there are a lot of things that I couldn’t seem to get to work. And then all of the sudden, it’s just like, “Oh, that’s what that third flag is that I thought I could ignore,” that thing turns this whole thing on. CHUCK: Besides some of the documentation issues – I guess it sounds like there’s some ordering things that you have to be aware of – do you find that people struggle with other things? Or, are those kind of the major things? JEFF: I think, sometimes, the struggle is realizing that what you’re doing in your program can even make use of it. A lot of people are thinking about their own programming in a certain way, and they’re thinking about their data in a certain way. They’re not realizing or maybe thinking about it in the sense of like if you have a bunch of data, maybe it’s 10 elements of data, maybe it’s a thousand, are there things that you’re doing to this data one at a time that you could instead want to try to crunch all of it as one block. I think what happens is, if you can think about it that way, you can start shopping around the functions that are in the Accelerate.framework and say, “Oh, you know what, I don’t need this function over here. But now that I’m thinking about it, I could use that to add a whole new feature to my program that I could even realize was a possibility.” Now, I’ve got a new thing that my program can do that my competitors didn’t think of because, again, they’re thinking about their data in a certain The easy ones are things like for getting programmers, they could be doing things where if you’ve got a massive multiplayer thing that maybe you’ve got a hundred enemies that you’re fighting, are you going around to each one of those enemies and having it do its little calculation for the AI to decide what it wants to do, like if it’s going to throw something at you. Are you doing all those individually? Is it your way to relocate your data for all of the enemies that are coming at you and do it using the [unclear] processing where you can do all these calculations at once. And then all of the sudden, you have a level of smoothness running in your game; maybe now you can do some things that you couldn’t do before because you didn’t think your game had enough horsepower to do more. ROD: It sounds kind of like functional programming, and you have to think a little bit different so you can write everything in just a different way to take advantage of everything. JEFF: Sure, yeah. It is maybe looking slightly oddly at something you’ve written before and trying to maybe see from a different angle. The nice thing is Apple is doing a lot of – they’ll kick in JCD if they need to, if they noticed that the amount of data you just gave it is appropriate. They’ll do things like they’ll just lay unrolling [unclear] to get massive speed gains. They’re doing all these tricks under the hood that is very time intensive if you’re on it to do it. JCD is pretty straightforward to do yourself, but you also have to think about your design pattern in your program if JCD is appropriate for the problem that you’re throwing at it. So it’s nice to have moments where you don’t have to think about that. I’m going to crunch all these data, and if JCD is needed, Apple figure it out [laughs]. CHUCK: [Laughs] JEFF: Let these routines handle some of these things for me. CHUCK: I’ve never actually taken the time to do much with Accelerate.framework, and I know that lstedI am probably neglecting to ask something. Are there things about Accelerate that we haven’t brought up that people ought to know? JEFF: Here’s something that I haven’t mentioned, and it’s also part of the win categories that Apple is making these things available, they migrated it from the Mac over to the iOS. If you are somebody who is developing for both sets of devices, they’re exactly the same calls for both of them. So even if your data is maybe another issue as another we got 64 Bit computing coming on the iOS side, there’s a lot of compatibility worries that are out there. This is one of those things where Apple is taking care of a lot of these for you. So if didn’t bother to learn maybe one function back 5 years ago that they threw in the program, you don’t have to worry about it now; you don’t have to worry about the new tricks that came out, you don’t have to worry about that you moved that whole code thing over to the iOS side. It’s just a lot of things that you don’t have to worry about. And the more you can have that in life, the more you can move on and not just rewrite the same data of code every time something gets updated. I should say, this isn’t something where you go through a program and you say, “I don’t know, I’m going to have to rewrite my whole program, rethink about everything.” There are times when using the Accelerate.framework really comes down to “I’m going to add these 6 lines of code somewhere in my program,” and that’s it. And you made your gain in the area where you needed to make your big speed gain, or maybe, you had a battery life issue. Somebody says they love your program that it grows down their phones when they stopped using it, and you’re like, “Oh, I need to really make this more battery efficient,” so maybe it’s a super lap where you just doing something visual and it’s chewing up the CPU; now, you can maybe relieve the CPU from using so much battery. But it’s all from just a few perfect places that you put it into your code. So even if you learn it once and then you just use this particular function so you’ve just dipped, you’re telling it to the framework, maybe then when you’re on the next project that you recon, you’re like, “Oh, this is another one does moments where you’re not maybe that exact same function, but I know there’s got to be something who just like it to do the [unclear] cast,” and then you drop in this magic gem just in the right place. Like I said, oftentimes, the code you are writing is more hideous than the code that you are avoiding by using the Accelerator. CHUCK: [Laughs] JEFF: That’s one of the things I tried to do with the [unclear]; I tried to show side by sides because unless somebody shows you it, it’s hard to believe somebody when they say that, especially there’s something called “The Convolution Integral”, graphics people tend to be more familiar with this. It’s how in the 80s, photoshop became photoshop. You would look at the pixel and you say, “I’m going to do something where I take maybe the average of all the pixels around this pixel, and then I’m going to get that value, and then I’m going to do it to every pixel on the whole thing. And then all of the sudden, you discovered that you made a blur effort, and you’re like, I’m going to write this and make a million dollars. Maybe, you slightly change that effect where maybe you’d just look at the pixels above it and to the right of it. All of the sudden, now you’ve got this slight shadow effect that you’ve created. So doing that operation to all the pixels in a grid like that, that’s called a Convolution Integral. Many kinds of Convolution things are built in to the Accelerate.framework. So basically, all of photoshoper up to some year is probably mapped pretty directly to one of these functions. You can use it not just for images, but you can use it for anything that has 2 dimensional data, and you can do it for bigger grids than just the 3x3 grid. You can do arbitrary relaunch grids, but it’ll slow down a few, pick up something to that. ROD: This would work great with machine learning because they’re doing all kinds of things with vectors and matrices. JEFF: Oh, yeah! Basically, if you’re cracking a math book at all, you should know the Accelerate.framework. So, if you’re into newer network stuff or you’re just learning that, that’s a great example. But like I said, a lot of times, people don’t realize that what they’re doing is Math-y. I think, the math people, the people who are into math, they know that there must be some fast version of a function that they need out there somewhere so they’ve probably stumbled into the Accelerate.framework. But it’s also people doing much more average programming who can take advantage of it. And it’s really having your own machine do some operation as this anyone knows how to right now. All the really hard part of that is being done for free by Apple, and they’re just hanging it out there for you to use. CHUCK: How did you find it? How did you find the Accelerate.framework? JEFF: One of those people who went to the session in 2004 and 2005 said, “Oh, my god! This is great!” I sat down for maybe 10 minutes with it, and I wanted to just check my computer out the window. CHUCK: [Laughs] JEFF: And it took several years until I actually sat down and said, “I’m going to learn how to make this silly little simple function work,” and then I did. I’m like, “Okay, I got this as simple as possible function working in,” then I just kept working at it. I started with what I thought was the easiest library – the BLAS library. And then I work my way up and got a little more bold, trying things out. I’d say for me, the hardest was vImage because I’m so what be handles with the graphics stuff. That was 2 of the most complicated. But I think, for people who do graphics stuff a lot, they might find out the easiest place to start because it’s already using the data types that they used for CoreGraphics. And a lot of people start with vDSP because if they’re doing any audio processor. It is worth repeating – Apple’s marketing on this. Something like 18 or the top 20 selling Mac apps around using the Accelerate.framework, so no coincidence there. It’s not just that you’re doing what you’re doing really well, fast and energy efficient, but you’re probably also doing effects that nobody else knows how to do because they’re not taking advantage of some pretty neat functions. CHUCK: Awesome. Alright! Well, we’re getting pretty close to our time, and I want to make sure that we get time for the picks. Did you have a question, Rob? Or, you’re trying to break in before I prep – ROD: I have one last question related to what he just said, and I was wondering, what are some of the audio things you can? Is it just DSP, or are there other things you can do? JEFF: There are hundreds of functions in there, so I’m not really sure what you’re trying to [chuckles] – ROD: Right. Well, just one example. JEFF: Well, there is a great effect Chris Cieslak doing; he was doing the ring modulator where you’d basically just amplitude, modulate, and input, and you just give it a sign wave so it’ll squash [unclear] over and over at a certain frequency. The funny thing is that, basically, what they use to make the sound of the [unclear] at Dr. Who… CHUCK: [Laughs] Exterminate! JEFF: Exactly. That loop is actually an ideal used case. There’s a function right in vDSP that allows them to take an array of data, multiply it by another array of data. That seems simple enough, and you’re like, “Okay, what would I want to use that?” Well, you shop around and there’s a different function that will turn an array of data into a signwave, all in one-liner code. So I make my soundwave and I multiply it by the audio coming in, and now I got a ring modulator. ROD: Cool! CHUCK: Interesting. JEFF: It’s mostly, once you get into it, you’ll look at the functions. If you start combining then, you realize the possibilities are pretty endless. And Apple keeps every single unless released; they have all sorts of new parts that they’ve improved and that divided functions. This latest dubdub, they had a couple of great announcements. I think it’s still NDA so I won’t say it, but anyways, it’s a lot of free win, and it’s ongoing, and it’s been done for you for free. That’s my pitch. CHUCK: It sounds really cool. I’ll have to go dig into it, and then maybe come back around to talk about it again. ROD: Yeah. JEFF: Hopefully there’s file I can get that set of examples, at least, to get people just try it. CHUCK: Yeah, that would be great. Let us know when you do that, and we’ll let people know about it on the show. JEFF: Oh, for sure! For sure, yeah. CHUCK: Alright. Well, let’s go ahead and do the picks. Rod, you want to start this off? ROD: Sure! I just have one pick, and somewhat related. There is a language out there called “F-Script” and it’s kind of like a small talk that’s built on top of Cocoa. The way it’s related is it has a lot of support built-in for array programming so you can treat arrays as units and do all kinds of neat things with them. That’s my pick – F-Script. CHUCK: Alright. I’ve got a couple of picks. I’ve been playing with a DevOps System called “Chef”, you can use it set up remote servers and deploy stuff to them. I’m really enjoying it. My first pick is a “Series of Videos by Nathen Harvey”, he’s one of the hosts of the Food Fight Show. He got together with a fellow who was looking for some help getting started with Chef. Anyway, what wound up happening was they got together and they did like a Google Hangout and work through the examples, and it really got me to where I need to be to be able to deploy some of the applications that I set up frequently for people. So I get request a lot to set up Rails apps since that’s what I spend a lot of time doing; things like Redmine or Discourse, or things like that. So I’ve been working on “Chef Cookbooks” for those, and like I said, it’s been pretty cool to just work through those examples. So my picks are Nathen’s Tutorial Series, Hosted Chef, and Chef itself. I’ll put links to those in the show notes. Hopefully, if you’re looking to deploy your server, that’ll get you well on your way. Jeff, what are your picks? JEFF: I got to say it’s very, very funny. I have the F-Script link that I was going to suggest as well [laughs]. I’m a heavy F-Script user, especially for Mac development until I consider it invaluable. And I’ll just add to what he was saying that you can inject into live or in code to F-Script. So if you’re debugging something, you can attach an F-Script object, browser, and console, and while your program is running, we can play with all the stuff live in the scripting environment, and see what effects it has. Sometimes, it can cut your debugging time [unclear]. But, am I allowed to pick the site that I run? CHUCK: Yeah! JEFF: Okay. There is a site that I run called “The Cocoa Literature List”. It’s not anything I do for any money there, no ads. It’s just something I did for myself when I made it available. It’s a site that indexes articles on programming in cocoa. Mac and iOS, it’s got 38 hundred articles that listed in there. CHUCK: Wow! JEFF: And it also allows you to search by content. So even if you’re looking for a function like if you put a functioning to Google and you come up with 10 Stack Overflow’s where half of everybody is doing it wrong. If you click the function name and hear, it’ll come up in the context of an article that somebody has written. So I find that, I use at all the time. If other people want to use it, it’s a good tool. CHUCK: Very nice. That sounds rally cool. I’d like to compare notes with you because I’ve thought about doing things like that for some of the other communities, I sort of so. JEFF: Oh, sure! CHUCK: Alright! Well, I think that’s the show! We’ll go ahead and wrap it up. It was kind of a short show, but I know that we had some time constraints among us, and needed to wrap it up pretty quickly. But thanks for coming, Jeff. It was really interesting to talk to you about this, and hopefully, this solves some problems for some people who were dealing with things that they kind of have a free resource that they can just go to. JEFF: Yeah. And if somebody wants to email me, feel free. CHUCK: Alright! JEFF: Thank you very much for having me. CHUCK: Yeah, no problem! ROD: Thanks, Jeff. CHUCK: We’ll catch you all next week!

Sign up for the Newsletter

Join our newsletter and get updates in your inbox. We won’t spam you and we respect your privacy.