030 iPhreaks Show – Building Hardware for iPhones with Joel Stewart
Panel Joel Stewart (twitter github) Andrew Madsen (twitter github blog) Jaim Zuber (twitter Sharp Five Software) Rod Schmidt (twitter github infiniteNIL) Charles Max Wood (twitter github Teach Me To Code Rails Ramp Up) Discussion 00:30 - Joel Stewart Introduction VP of Engineering at Canopy Video Game Development 01:06 - Building Hardware Sensus Apple’s MFI Program 04:34 - Connectors 09:11 - Challenges of connecting a device through a lightning adapter Case Certification CES 11:39 - Build Process 17:24 - Detection Sensus SDK Developer Portal 21:54 - Bluetooth 4.0 Pebble Smartwatch 25:12 - Security 26:59 - Development Interface 29:22 - Sensus Release Market Strategy Leap Motion Picks i-calQ (Rod) Flow DJ Software (Andrew) MIKMIDI (Andrew) Bluegiga (Andrew) QONQR (Jaim) Airbnb (Chuck) Delta Airlines (Chuck) CARROT (Joel) Harvest (Joel) Next Week High Performance Core Data with Matthew Morey Transcript ROD: The audience is listening. CHUCK: Hey everybody and welcome to Episode 29 of The iPhreaks Show! This week on our panel, we have Andrew Madsen. ANDREW: Hi from Salt Lake City! CHUCK: Jaim Zuber. JAIM: Hello from Minneapolis! CHUCK: Rod Schmidt. ROD: Hello from Salt Lake! CHUCK: I’m Charles Max Wood from DevChat.tv. This week, we have a special guest and that’s Joel Stewart. JOEL: Greetings also from Minneapolis! CHUCK: Joel, do you want to introduce yourself for those of us who may not know who you are? JOEL: Sure! I’m currently a VP of Engineering at a company called Canopy up in Minneapolis. We’re a startup that is focusing on iPhone accessories. We do everything from hardware all the way up to stack to applications, then we part with a lot of content providers to integrate our functionality with our hardware inside their applications as well. Me personally, I’ve been doing the new game development for the last 6 or 7 years and I recently gotten involved in hardware. And yeah, [that’s] about it. CHUCK: Awesome. So you’re going to make a video game controller for the iPad? JOEL: Oh, yeah. CHUCK: What kind of hardware things are you building? JOEL: Well, our first product is called Sensus, and it is an iPhone case. It adds pressure sense with multi-touch panels to the back of your iPhone and also the edges. So, it can rename with functionality such as squeezing to scroll as opposed to using a thumb on the front of the screen to drag the contents around; you can just squeeze the edges and the content will automatically scroll. We do a lot of other things such as moving that touch and drag functionality to the back, moving environments as oppose to characters in the front; just adding an extra layer of interactions for your applications, for your games through the hardware. It’s attached to your iPhone through the Lightning connector right now, and it is part of Apple’s MFi program, which is made for iPhone. You have something to plot for and it has the entire slot for documentation that general public typically doesn’t get access to. CHUCK: Interesting. Is there some kind of trick to building hardware for iOS devices? JOEL: Certain metric. It’d be similar to most hardware products you create. You have a lot of industrial design (basically the plastics), a lot of mechanical engineering which is identifying what materials go into it, electrical engineering laying up those boards, firmware development, software development. I know it’s a much larger undertaking in just writing software, but process is a lot longer, it’s a lot more expensive, and it’s not easy, that’s for sure. ROD: Do you have to provide a driver to go with the hardware? JOEL: For the most part, no, since we’re talking through Apple’s Lightning connector; most of that protocols are already established so they provide a lot of documentation as to how to talk to the core operating system, iOS,
ROD: The audience is listening. CHUCK: Hey everybody and welcome to Episode 29 of The iPhreaks Show! This week on our panel, we have Andrew Madsen. ANDREW: Hi from Salt Lake City! CHUCK: Jaim Zuber. JAIM: Hello from Minneapolis! CHUCK: Rod Schmidt. ROD: Hello from Salt Lake! CHUCK: I’m Charles Max Wood from DevChat.tv. This week, we have a special guest and that’s Joel Stewart. JOEL: Greetings also from Minneapolis! CHUCK: Joel, do you want to introduce yourself for those of us who may not know who you are? JOEL: Sure! I’m currently a VP of Engineering at a company called Canopy up in Minneapolis. We’re a startup that is focusing on iPhone accessories. We do everything from hardware all the way up to stack to applications, then we part with a lot of content providers to integrate our functionality with our hardware inside their applications as well. Me personally, I’ve been doing the new game development for the last 6 or 7 years and I recently gotten involved in hardware. And yeah, [that’s] about it. CHUCK: Awesome. So you’re going to make a video game controller for the iPad? JOEL: Oh, yeah. CHUCK: What kind of hardware things are you building? JOEL: Well, our first product is called Sensus, and it is an iPhone case. It adds pressure sense with multi-touch panels to the back of your iPhone and also the edges. So, it can rename with functionality such as squeezing to scroll as opposed to using a thumb on the front of the screen to drag the contents around; you can just squeeze the edges and the content will automatically scroll. We do a lot of other things such as moving that touch and drag functionality to the back, moving environments as oppose to characters in the front; just adding an extra layer of interactions for your applications, for your games through the hardware. It’s attached to your iPhone through the Lightning connector right now, and it is part of Apple’s MFi program, which is made for iPhone. You have something to plot for and it has the entire slot for documentation that general public typically doesn’t get access to. CHUCK: Interesting. Is there some kind of trick to building hardware for iOS devices? JOEL: Certain metric. It’d be similar to most hardware products you create. You have a lot of industrial design (basically the plastics), a lot of mechanical engineering which is identifying what materials go into it, electrical engineering laying up those boards, firmware development, software development. I know it’s a much larger undertaking in just writing software, but process is a lot longer, it’s a lot more expensive, and it’s not easy, that’s for sure. ROD: Do you have to provide a driver to go with the hardware? JOEL: For the most part, no, since we’re talking through Apple’s Lightning connector; most of that protocols are already established so they provide a lot of documentation as to how to talk to the core operating system, iOS, as well as into application that’s using the external accessory in session protocols. So most of the time, they’re written for us. ANDREW: You must be in the make for iPhone program, which do use the Lightning connector, right? JOEL: Yes. Yup! ANDREW: So that’s something that’s not really available to everyone; you have to apply and get accepted? JOEL: Right. It used to be very restrictive. I was getting to the MFi program 5 years ago; it was not something just anyone could apply for. I think it was about 3% of applicants who’d actually get accepted into the program. Nowadays, they’ve open to do that quite a bit; you can actually just go do that when you get to apple.com/mfi and apply for the license there. They usually just look for a business entity behind everything next with all your documents in the right order and everything. But then, that’s step 1. Step 2 is you actually have to snip a product plan to them saying, “Here’s the product I want to build, it’s going to use this type of connector, here’s what it’s going to do, etcetera,” and they’ll either approve or reject that product plan based up what other product plans are in the market. It’s a great way for them to make sure that the market just not gets slaughtered with copycats. They preferred just one company making a singular product. That kind of helps vary the entry for competitors. It’s a great ecosystem to play in that regard. Does that make sense? ANDREW: Yeah, that does make sense. I know somebody who applied for that program, probably 3 years ago, and they have been had switched a great experience… CHUCK: [Chuckles] ANDREW: But it sounds like it’s gotten better now. JOEL: Yeah, we applied for it, I think the original company before Canopy was called Canopy, they applied for it. I think the better part of 4 years ago and they got in, and then we had to do a reapplication about a year ago. That went through pretty easily. ANDREW: So you used the Lightning connected. Does Apple give you a high level API to talk? Or, you’re using sort of low level stuff? This is all kind of hidden and locked sure in the program, and I’m so curious about it. JOEL: I’ll try and share, I won’t reference any particular documentation, but it’s just kind of more general stuff because there are certain NDAs in place whatever… ANDREW: Sure. JOEL: You talked over IP and you actually get a very specific type of connector. There is few variance; the connector determines whether the phone has been talked over USB host mode or USB device mode or a serial connection. The connector determines that if you ever ripped open when the Lightning connectors are looked at some of the disassembly diagrams, you’ll see that there’s actually a large amount of circuitry into that actual Lightning connector. That tells the phone how it’s going to communicate over this connector, and then it’s up to you to implement the proper stack or anything, whether it’s a USB stack or just a simple serial connection. ANDREW: Cool! CHUCK: Do you have any thoughts about maybe doing a device that connects through a Lightning connector or Bluetooth? Or, are they usually completely different devices that do Bluetooth? JOEL: Typically, you’ll choose one of the other. It’s a talk over Bluetooth, if you want to use something BLE Bluetooth or energy for quite old stuff, you actually don’t even need the MFi approval, and you can do that outside of the Apple’s program. But if you’re using anything 2.0 or 3.0, the more traditional Bluetooth, you have to go through Apple’s program. But otherwise, typically we won’t. Right now, the overhead of adding on a Bluetooth chip in that communication is pretty substantial, both in terms of engineering efforts and the cost materials, etcetera, because you have to add thing – we’re talking about the battery; there’s a lot of other certifications you have to go through as soon as you start emitting wireless signals. It’s a large, about a whole, since you start down that road. ANDREW: Well, with your application with the case that’s physically surrounding from Bluetooth, it doesn’t seem to make as much sense. JOEL: Yeah, not at this time. ROD: Seems like a lot of companies go to Bluetooth routers. Is there any reason for that? CHUCK: I was going to ask that question in different way, and that’s as: what types of devices [unclear] sells better to one or the other? JOEL: I don’t really know the concrete answer to that [laughs]. CHUCK: [Laughs] JOEL: I ain’t got stock for interpretation. I think there are a lot of game controllers that are coming out in the next 6 months to a year that will be using Bluetooth ally or Bluetooth. I’m not sure why they chose that route as oppose to using a Lightning connector. It’s a talk over Apple’s MFi game controller APIs; you still have to plot that MFi license, which would give you access to the Lightning connectors, etcetera. But I’m not sure why they went to use the Bluetooth chip as oppose to Lightning. The Lightning connector does carry a pretty high cost and it’s not cheap because of Apple bits of royalty into the every single connector that they sell to you. And the [unclear] is possibly more expensive than the Bluetooth if your battery is like AAs or AAAs which I think some of the game controllers coming out or using that and kicking of thought. The mobile when I think is these batteries and AAAs. It could be wrong. ROD: One reason that they might choose to use Bluetooth is so they can talk to the Apple TV. JOEL: That’s also a thought, yup. ANDREW: At some point [coughs]…not now. CHUCK: [Laughs] I was going to say. ANDREW: [Laughs] It doesn’t mean now that you can talk to Macs with modern Mac solver which is already built in. They obviously don’t have Lightning connectors. JOEL: Right. JAIM: I think that it really only comes to what agency do you want to deal with. Do you want to deal with Apple? Or, the US government? Either stuff is okay. [Laughter] JAIM: Pick your poison. ANDREW: There are actually Bluetooth – it depends on your application, of course – but there are these Bluetooth already in modules that you can buy that already have the certification. And as long as you don’t change the RF characters that’s of those file, for example, changing the antenna, you don’t have to get your own certification. JOEL: Yes, sure. Yup! ANDREW: So Bluetooth doesn’t have to be terribly expensive. Well, of course if you pay extra for those pretty engineered modules, but it doesn’t have to be a terrible, regular, torrid nightmare. JOEL: In the US, depending upon the country, you may have to go through retesting depending upon how significant the kind of foot print that device is having. For instance, Australia has one of the more restrictive certification processes. It’s maybe that if you buy one of those octopus-felt products, you’d still going to have to go through a fair amount of strict testing with that. ANDREW: That’s a good point, yeah. Having to deal with the loss in a bunch of different countries is not necessarily fun. JOEL: Nope. CHUCK: So what challenges have you run into connecting a device through the Lightning adapter? JAIM: Smooth sailing, that’s what I’m hearing. CHUCK: [Laughs] JOEL: Not too much. I think the biggest challenges have to do with the design of how we actually get a case to kind of have a retractable Lightning plug, trying to drop the phone in and insert it, and still declassified as a case. So Apple has a lot of different certification process that you go through, both from MFi to RF interference to what they just call general case in certification. And their Case Certification is kind of the most difficult one; it’s what you have to go through to make sure that you can get your product into Apple retail. And some of their requirements are very vague; it’s just very black box that you throw over the fence. They may say, “Yes, you passed,” “No, you didn’t,” and they won’t really tell you why typically. So you basically have to work with Apple and some of the people on the outside of that box to say, “Here’s what we’ve seen failed, here’s what we’ve seen passed. You should probably avoid cases that have this type of design, etcetera.” And one of those is what they call basically slider. That’s any part of the case that has sort of like friction or movement along the phone. Typically, those will get rejected. So if you simply go in to Apple retail store and look what’s on a shelf, you’ll very rarely see a case that has in-stored of sliding aspect to it, which makes it really difficult to create a form-fitting closure that has a Lightning plug that will be inserted into it. That’s going to be one of the most difficult challenges for the last year. So you’ve designed to this coming up with that industrial design that really works well from a consumer’s stand point, that’s easy to insert and remove that plug and still follow within the crazy unknown restrictions of Apple. CHUCK: That makes sense. I don’t’ know if you’ve been to CES, but they have a whole room, and by room, I mean like a whole convention room [laughs] full of booths selling iPhone cases and accessories. So can I assume that most of those don’t go through the process of getting approved, does an Apple case? JOEL: Correct. Yup, most of them will not get approved. And they don’t even bother going through that process because they don’t really care to be in Apple’s retail channel. And yes, we are at CES last year, and we bought this [unclear] case waste land. CHUCK: Yeah, we walked up about 3 isles, and we were like, “We’re going to go to the other room,” [laughs]. JOEL: [Laughs] CHUCK: I might make it out this year, so I’ll keep an eye out for you guys. JOEL: Very good! CHUCK: So walk me through the process. Let’s say that I have this brilliant and original idea that I’m going to make an alarm clock with speakers on it that I can plug my iPhone into the top of. JOEL: That’s a brilliant idea. CHUCK: I know, and it’s totally unique. Nobody has done this before. JOEL: [Laughs] CHUCK: So what’s the process that I go through to build my product assuming that I have a guy or team of guys that know how to make alarm clocks so they can design the display and the speaker and everything? JOEL: Sure. Well, if you’re going to talk over to Lightning connector, the first thing you want to do is to get that MFi license so you don’t need to go through [unclear] and get all that business legal entities stuff squirted away your dumbest number, whatever that is. Make sure that you get the license from Apple. I guess that, nowadays, I think it’s a lot more open so it’s like you’ll get it. I know you have to submit a product plan which describe to what you’re trying to make, and that’s all on your end. And then the product plan approval can take anywhere from 1-5 weeks, maybe 6 weeks. You’ll be assigned the representative within Apple’s MFi team that you’ll be in contact with. They’ll ask you questions and they’ll look for any sort of red flags on their end, making sure that you’re not trying to do things that they don’t’ want you to do such as attaching the big old hard drive so you can store music files on it or something like that. Well, I think they might even allow that [chuckles] everybody’s [unclear]. So once you have your product plan approval, then you can start ordering the Lightning connector components and the authentication chip components from App.net, and the bright of special portal. You’ll pay a hefty price for those components because they’re kind of just small orders, small sample quantity of these components that ain’t get to work on both on your hardware basically. You’d get access to about 500 pages of documentation on the protocols that they use and how to all the different features and functionality of the core iOS that you can use to get access to the library, to the music library, initiate playback, get the audio left drive channels coming through the Lightning connector, etcetera. And then once you’re in development, a lot of it takes some time. You’ll start, once you’ve nailed it down and you get ready to start going through all the certifications, you’ll start submitting self certification reports to Apple through their systems that basically prove that you’re implementing their stacks correctly. And then you’ll have to ship their MFi team kind of the developer units so they can actually see it and play with it and test it and make sure that it’s working correctly, and then you’ll eventually send 1 as well for testing whether it’s RF testing if it needs it. There’s a swap whether it’s certifications you may fall into depending upon the type of product. I know it’s all that’s done; you’ll have try that basically. ANDREW: With this case that you are making, you guys probably relying on third-party developer’s integrating support for write it something that apps can use, is that correct? JOEL: Yes, that is part of our marketing strategy. We will be developing our own applications specifically for compatibility. We’re going to be building up those apps over the next 6-9 months, and we try to release maybe 10-15 of our own, getting some pools and set we haven’t ever been done before. Like for instance, I can’t guarantee what’s going to come out, but I’m pretty proud that we’ll get it out there. It’s actually like the scale. So if you turn your iPhone upside down, other table are in place and object on top it will tell you how much it weighs. It was like that. CHUCK: Oh, interesting. ANDREW: It sounds cool. JOEL: And then we have a large market that is kind of off the radar for the most consumers, which is tailored hard of object there like the vision impaired market, so blind people or mostly blind, vision impaired people. If you ever try to work within an iOS application, your eyes standpoint are accessibility’s standpoint, it can be kind of a nightmare; depends it on how much effort the dev is splitting to making their application accessibility friendly. They may correctly label the buttons or all these new things, but there’s an element of zooming and moving around into navigation, but it’s really difficult for the average user. VI people have gotten used to it, but there’s still a lot of room for improvement. And Apple’s IP2 protocols, which is the iPhone accessory protocol, it’s what we use to talk over the last – it has a lot of accessibility functionality that we can tap into, so we’re going to be adding a lot of cool things to target that marketing in particular and improve their life significantly. One of them is probably going to be like a Braille keyboard , which will actually going to have about 6 points of contact on the back and the top edge, and just by squeezing those point slash, they’d be able to type in Braille as opposed to doing little hand pack functionality in front of the screen, etcetera. The average kind of market is going to be there. We will be leveraging third-party content providers to help – it’s kind of win-win situation for everyone. As consumers each bought a device, they’re going to be looking for a new applications to use with their device so it’s kind of reset on a new market. So if you are a small indie developer and you want access to the 500,000 consumers who add a Sensus device in the first 3 months, it’s going to be a really small pond to plan and we can get a lot of visibility a bit quickly. If you’re a large-scale-trip-away-marky type partner, you’re going to remind as high or add functionality or get more brand recognition or placements under the opportunities with Canopy to throw over a level like bots, all those different types of things. We have a large idea of how we’re going to leverage content providers in our early placement in Apple retail stores to try value to pretty much everybody involved. JAIM: If I’m like an indie developer and I want to detect, Sensus is the case? JOEL: Uhm-hmm, yep. JAIM: So I want to detect that Sensus is plugged in, and let’s say I want to use the back of the case as like a ScrollView so like in detect jesters on that, how would I do that? JOEL: You have to sign up for the SDK on our website, developer.getsensus.com, and it’s a quick integration. Basically, you’d drop it in, you’d put in your info [unclear] say, “I can talk to this type of protocol or talk to this accessory that speaks as protocol,” and then there’s the typical one-line initialization that you put in your AppDelegates, it starts it up, and then you can just start subscribing to touch events just like for normal touch events. We’ve built in pretty much similar objects. So a Sensus touch, looks, and behaves just like a UITouch but it has a much nicer data in there as well. A little go up the responder chain on UIView stack as well and create the gesture recognizers just like the built in ones for the friendly advice, except these ones that are targeted for the back. And then they have a lot of different helper objects like edge wheels and different things that you can use to basically drop and create extra – it’s a very easy way to add extra functionalities to your app really quickly out of blind testing because DevKits are going out shortly, but most of their third-party developers have not had access to our DevKits yet. So we’ve had some that it come through and it manage the SDK without actually having the hardware on hand. The major ship is in IPA and we play with it, and it has worked great. You can see some of the games online and then videos and what not. It’s pretty straightforward. It’s very simple. JAIM: So what’s the process for defining a newly API for these new gestures? JOEL: From the third-party? Or, for me with the SDK writer? JAIM: For the SDK writer. JOEL: It’s a lot of work. The gestures are actually then one of the more complicated parts of the system because we wanted to mimic and feel just like Apple’s gesture recognizers, now it’s familiar. We put a lot of effort in that. Those actually aren’t released right now in the current SDK that you download; those are becoming in a few probably in about 6 weeks. So we’ve been working on that pretty heavily, and it’s a lot of work. ROD: Are you able to integrate your device so you can use gestures on the spring board to launch apps and stuff? Does Apple give you that kind of access? JOEL: We do get a fed access, but we’re being very careful about how much core or less interaction we actually do. We use just by the nature, that you’re actually holding the device in your hand; there’s going to be a lot of will be classified as noise. How do we isolate and detect what is attached and what’s just you holding the device, which is actually why we moved to a pressure sensory technology as opposed to a capacitive touch tech. So our DevKits to date are demo units, have all been cap touch, which is the same tech as on the friendlier phone. The ability for that technology to really isolate and detect will be noise or accidental or non-real touches is very limited. So we moved to a pressure tech, which it works almost identified with cap touch from the user’s perspective that our ability to detect and say, “And not just I’m holding it,” that’s kind of a grip decision or that’s just a light touch that’s really not intentional. It’s really exempted and do a lot more millions detection. But even with that extra millions detection, we don’t want to have you activating or doing things to the really last navigating spring board or launching apps, unless it’s really intentional. So we add a lot about adding really complex constant gestures like squeeze the device really hard for 3 seconds to launch Evernote or Safari or something. We do have the abilities do that, and that is something worth-exploring when we add that newest feature. So, it kind of like a fast app switching idea. ROD: That kind of access comes along with the MFi program? JOEL: Yup! ROD: Can you tell us what other kinds of things you can do? Or, is that under NDA? JOEL: I’m trying to think of what else would be kind of…I know when you keep word input is an option. Airplay functionality, WiFi, way in discovery basically. I don’t know, there’s 500 pages, so [laughs]…yeah. CHUCK: [Laughs] JOEL: [Laughs] There’s a lot. They opened up a lot. CHUCK: Dramatic reading next week. JOEL: That’s right. That’s right. CHUCK: [Laughs] So is there any way to get that information without actually being in the program? JOEL: No, technically not. ROD: If they opened up that more, it would really change things. ANDREW: There’s a lot you can do now with Bluetooth 4.0 especially. It used to be that unless you’re made for iPhone program, you really didn’t have any option other than WiFi. I guess people figured out how to use the headphone jack to send data using audio, which is low beta rate. But I think Bluetooth 4.0 has opened up a lot of possibilities that I’m curious, Joel, if you have, maybe outside of the work around Sensus, if you have any experience with the sort of newish Bluetooth 4.0 stuff that’s coming up? JOEL: A little bit. I definitely gotten to other sessions at WWDC this year. There’s some little demo units that we have floating around the office that we’ve been playing with. So we’ve done a little bit but nothing with like metro project; mostly just tinkering. Yeah, the BLE’s factors are really interesting. It’s not high foolproof, but it is fast. It’s great for like game controller type thing, or those small amounts of data just got re-services when dropped those iBeacons around here retail stuff, those types of serials that it’s great for. ANDREW: I’ve played around with it a little, but I also haven’t worked on a real project but just for fun stuff. There’s a lot even do with it; it’s certainly not the same through or put as you might get on the Lightning connector with USB or WiFi, but it’s very easy to get up and running. I think we’re saying that because there are a lot of accessories that are sort of being announced and coming out. They are using Bluetooth 4.0 and some of those can’t really be done with other technologies like these Bluetooth 4.0 deadbolts. I think there are like 4 companies making deadbolts that use Bluetooth 4.0 to detect when you’re in your [unclear] and unlock the door. CHUCK: That’s pretty cool. ANDREW: Anyway, I’m excited about iPhone hardware possibilities compared to even just like 2 or 3 years ago. CHUCK: You don’t want the Lightning connector that deadbolt plug your phone into your door and then have it open? ROD: [Laughs] That might be safer [chuckles]. ANDREW: It might be. JOEL: There is a WWDC session this year on iBeacon and BLE. They basically showed how they’re going to develop the iWatch, which is basically they put a push notification service on top of BLE. So it actually have a slide in there that actually shows a watch, basically showing all your notifications and giving you a functionality communication back to the iPhone from the watch all over BLE, and other service they built on top of the BLE stack. I don’t know if that’s eventually [unclear] while we look at that. I forget what session it is, but it’s definitely really interesting to see how Apple, internally, are already leveraging that to kind of pave their road for upcoming devices. ANDREW: Yeah, I was at that session. I think it was interesting to see some of that stuff because I think, you’re right, that they give a hint as to what Apple is taking about internally. But cool! The thing is, everything made sure in that session is maybe available to people outside of Apple. JOEL: Right. ANDREW: So there’s actually a lot you can do. JOEL: I’m actually surprised we haven’t seen more accessories take advantage of that. I would have figured that everybody have seen a lot of accessories out already to do some of those cool things. I think Pebble, the Watch Pebble, just the recent update, I think last week, to do that type of integration. ANDREW: I’ve actually got my Pebble on right now, and they did, just in the last week, an update. So now, you can get all of your notifications; anything that’s in notification center is pushed to the watch. JOEL: There you go. CHUCK: One other thing that I’m curious about with a lot of this stuff, is there any way to compromise the security of the phone by plugging something into the Lightning cork or by pairing the phone to it over Bluetooth? Do I have to grant access to the device when I plug it in? JOEL: Yeah, you do. Now, you do. In iOS 6, you didn’t; in iOS 7, it’s going to ask you if you want to trust this accessory. I’m not sure what permission it’s going to ask you, like if it’ll ask you to – because you can’t get at contacts, you can’t get at library or your music library, and all that stuff. I’m not sure if it asks you for each individual permission or not. CHUCK: Yeah, it’s interesting. So if you leave your phone unlocked and you plug something into it or somebody else has plugged something into, then they could compromise your phone? JOEL: Yeah. Well, there is that whole man-in-the-middle problem that cropped up from my close charging stations. I don’t know if you’ve heard about that, but they had basically put in a computer behind these charging stations like at the airport, like the quick charge, when you plug in your iPhone, it’ll basically just ripped through your iPhone and grab all of your contacts as much data as it can grab over these protocols, and then store it in [laughs]. That was much more aggressive. CHUCK: Yeah. JOEL: Anytime there’s any sort of data connection, the iPhone asks you if you trust the other end. CHUCK: Does that mean…well, I guess it depends on what the Sensus is capable of, but do you have to build in then security protocols on the Sensus to protect the security of the phone once they trust your accessory? JOEL: Not really. CHUCK: Because you don’t talk to the outside world? JOEL: Yeah, we don’t talk to the outside world, yup. CHUCK: You just keep track of when somebody pushes something on their phone? JOEL: Basically. JAIM: So what is the development interface to like the Lightning connector? Are you doing that in – is that in C? JOEL: Yes, they’re mostly written in C. Our microprocessor is just an arm chip, and it got some other things down there to facilitate all that information. But, it’s just an arm chip, and we talk at all our USB over IP2, etcetera, so just bots of stacks. JAIM: Have you done much C before developing the embedded stuff? JOEL: Yeah, I’ve done enough to where I could get my hands dirty. Probably there’s so much going on and I had to bring in some help to get some of that framework done and written faster. So we do look at consulting and contract work for kind of early development, and then recently, I actually brought in a fulltime edit engineer to help out to that development. So we have a lot of features on the road, we got plenty to get done here in the next few months, so we’re actually looking for other C engineers. So if anyone here in Minneapolis – JAIM: Okay, it sounds good. So you have done Objective C before that, where are some of the gotchas that people may not realize going from Objective C to C? ANDREW: I think Joel is talking about actual firmware development, which is a whole kind of different world anyway. JOEL: Yeah. There is no object oriented premise to C. The memory management is much more heavy than just throwing a rotating releases on Objects, although if ARC, you may have to do that and more. But yeah, you’d do a lot of allocation and frame out of memory, and it gets really constraint because you’d only have a few kilobytes and memory to work with that fills up pretty fast. JAIM: Oh, definitely. No, I asked because I start doing embedded C way back when, so going to Objective C wasn’t a problem, and going back down to lower level, I can understand that. But if you’ve been working on the Objective C world where you have objects, stuff like that, I’m just kind of wondering what the reverse process might be like. That’s why I asked. JOEL: Yeah, I’m in the same boat. I did C and C++ early enough to where it wasn’t like I was going relearn something new, or it was just kind of like, “Oh, yeah, I did this a while ago.” CHUCK: I love teaching people C and watching their head to explode when they’re trying to figure out how many stars or app presents put in front of something. [Laughter] JAIM: Or, teaching them how to do Objects with C. CHUCK: Yeah [chuckles]. ROD: You do in more than 2 asterisks? [Laughter] ROD: Really confusing. CHUCK: Yeah. ANDREW: [Laughs] Yeah, like there’s all the way down [laughs]. ROD: [Laughs] JAIM: So when can we get the Sensus? When can we see this? JOEL: We are in the middle of manufacturing all of our DevKits. Plastic is being tooled right now, and for a shot, it should be in a few weeks. And the boards are being printed and set it right now. So we expect everything to come together end of November or the first week of December, and we’re printing about a thousand DevKits for release over the course of December and January. And we should be doing a few Hackathons in December, it was the current plan. So we’re taking a few shows in probably the major cities – New York, Chicago, San Francisco, maybe Bloomington or Denver area. And then we’ll go back and we’ll be doing a few in Minneapolis as well. So definitely, keep your eye on our portal, on the development portal, getsensus.com, sign up for newsletters, etcetera, and we’ll be coming to a city near you hopefully, and [unclear] our DevKits. And also you can apply for DevKits on website as well, and hopefully you would get in the first round. If not, you’re most likely get in the second round. It’s like around where it’s still planning, but it’ll be thousands of units that we’ll be shipping out [unclear]. And then, we’re looking at consumer retail launch; it’s still up in the air, depends on how fast we can make them, and a lot of other certifications, etcetera. But hardware is hard, just like I said, it sounds like, “Yeah, I’ve finished my products. Now, I can just to read it to a million people on the App Store.” It’s actually, we have to go through months of manufacturing and certification, and it takes a long time. So sometimes, it’s hard to predict. CHUCK: It sounds like a really interesting process, though. How do you go about marketing something like this? You talked a little bit about the Hack events and things like that, but how do you get the word out that people can get something like this and some of the capabilities that it has? JOEL: I talk mostly about the consumer market. VI market is kind of emceeing to its own, so we’ll leave that alone. For the consumer market, we were at CES last year, probably a little too soon, but at least, good; we had some good publicity. And then we do go to all like the developer conferences, so GDC, ADC, CES (again, this year, we’ll be there with just like a meeting room now to light up kind of this small individual interactions with publications and things like that). But otherwise, the market strategy is get into retail and get in front of the consumers however you can. So we will be looking that existing content providers to help, going to be our sales team as well. For instance, you have a game that has 10,000 daily active users, and even it vary with the SDK into your application and you want to maybe re-monetize or make some more money off of your consumer base while offering something substantial for their money. Our SDK will include kind of a separate part, which is the SDK to the Sensus UI (it’s what I’m calling it). You can visit the job then in order form directly into your application saying, “Hey, my game (or my application) works better with Sensus. Are you interested?” and they can watch little videos, see it in action like, “Yeah! That’s great!” [unclear], and then we’re going to give it a very large commission to you for basically selling Sensus directly to your consumer. Anything from $5-$20 is on the table right now, so it’s a pretty large number for app developers to find a new way to monetize their applications and their users while at the same time not be like it’s a premium plane or just bulging them for coins. That’s kind of part of our strategy. Details are still being worked out on exactly what that red new is going to look like for the app developers. It’s going to be significant I can say. It can really that. JAIM: What type of developers should jump on this opportunity? Give us some ideas. JOEL: Everyone [laughs]. CHUCK: [Laughs] JOEL: Honestly, everyone. JAIM: Specifics. JOEL: Game developers for sure, kind of the no-brainer. JAIM: The ScrollView, you give to it one hand from a finger, from like index finger in the back of your hand without getting to your hands in there like any app can use. JOEL: Yes, simple functionality to really heavy integration; anyone and everyone in between. CHUCK: That’s one thing that I can kind of see this going toward – the power users on the iPhone. Where, they basically can setup some kind of custom gesture on the back or the front or the side or they can touch it in certain places and certain ways, interact with the phone in new ways and have it do common things for them that aren’t necessarily built in to the app where you can only touch the front of the screen. JOEL: Yeah. We have a few easy winds in situations like that or it’s like squeeze-to-scroll or the scroll on the side like a Blackberry scroller. Those are kind of obvious no-brainers, but we look at it as kind of new playground, mostly people are happy to deal with the weak motion which is all in for our cameras and hand waving gesture recognition. It’s kind of this new sandbox in your playground to work with. We’re still kind of in exploratory phase but sleep in some of all those other devices, and even going to have that same kind of period with the Sensus. You’re going to have the kind of abstracted extra touch and put panel, and how do I interface with my apps. And much like our touch interface has evolved on the front of the device when Apple released iOS, there wasn’t a gesture recognition, it was just touches; the main add of a gesture recognition. And they added the “pinch to zoom” scroll, things that we’ve never thought of necessarily and we still continue to see iterations on that. There’s a great browser called Dolphin browser for iOS that is actually very gesture recognition heavy, really interesting. So I think we’re going to see a lot of new interface design come about because of Sensus and it’s going to be really interesting to see what people do with it and what gets adapted by the community at large. CHUCK: Very cool. I think you should do one of those launch events in Salt Lake City, just throwing that up there. JOEL: Salt Lake City? CHUCK: Yeah, you can think about that. JOEL: Okay. CHUCK: Are there any other aspects of this that we didn’t ask about because we probably just didn’t know to ask about it? JOEL: Well, we covered a lot! I guess my closing comment would be: Don’t be afraid to play for ad like a raspberry pie or something and for on there the BLE modules and start having some fun. You’ll never know what ideas you come up with and what may turn into a very viable product, and then kickstart at something silly. CHUCK: Sounds like fun to me! ROD: If you want to play like that, it’s also be in MFi program, don’t you? Unless you do Bluetooth I guess… JOEL: Yeah, you could do the BLE then. You don’t have to be in that program. ANDREW: There’s actually a – if you’re just playing around and not trying to develop product you’re not, at least, maybe at the start – there’s actually a serial cable you can buy from Redpark, maybe that should at one of my picks [chuckles]. Anyway, the serial cable, you can buy – they’re actually made for iPad programming. You wouldn’t be able to sell your app on the App Store, but just for getting your phone talking to hardware over a cable, that’s a super simple way to do it. JOEL: I forgotten about that solution. But yeah, that is definitely an option. It kind of provides that interface to talk to your application through external accessory session, and then use that to implement the serial protocol on your entry. It’s pretty easy. ANDREW: There’s not really anything easier than serial on an embedded device. JOEL: Yup! CHUCK: Cool! Alright, well, I think we’re getting close to the end of our time so I’m going to go ahead and start to wrap this up. But thanks for coming, Joel. It’s been really interesting to talk about this and hopefully, we can inspire some other ideas of just some really cool stuff that you can do with your iPhone and possibly even with your Mac. JOEL: Uhm-hmm. CHUCK: Let’s go ahead and do the picks. Rod, do you want to start this off with picks? ROD: Sure!! I just have one pick today. I saw on KSL, our local news station a story about a doctor who invented a app and a device that is being compared to the Star Trek tricorder. It does diagnosis that you take a blood sample, and then you plug it in to this device which is attached to your Smartphone and basically it uses the camera to analyze your blood and tell you what’s going on in about a few minutes. So that looks pretty cool. That’s my pick, it’s “i-calQ”, and I’ll put the link in the notes. CHUCK: Awesome. Andrew, what are your picks? ANDREW: I have 2 picks today. A couple of self-serving, but Mixed In Key just launched an app in the last couple of days as we record this, and this is what I’ve been working on for a year and I’m really excited that it’s out. It’s called “Flow”, and it’s our take on life performance DJ-ing so this is an app that DJs will get. The big feature is that it’ll smoothly beat, match, and mix between portions of the same song. And then as part of my work on that – so the app supports MIDI controllers – and going along with our topic today, I actually open sourced. We open sourced the framework we wrote to make it easier to talk to MIDI controllers, and it’s for both Mac and iOS, and it’s called “MIKMIDI”, it’s on GitHub. And then my last pick is “Bluegiga” which is a company that makes Bluetooth LE modules and they make modules like I talk about there off the shelf that already have regulatory plans. They have been certified so you can use them in your products and they’ve got DevKits that you can use to get up and running with Bluetooth LE and an iOS device quite quickly. Those are my picks! CHUCK: Awesome. Jaim, what are your picks? JAIM: Alright, I’m going to kiss up to the guest a little bit. I’m going to pick a game that’s been on for a year or 2 called “QONQR”, Q-O-N-Q-R. I actually met Joel because he had done the iPhone version of this game. It’s a game that was done by some local guys from Minneapolis and they’re all kind of .NET people. Joel helped them out with the iPhone version. It’s kind of a risk on a world stage, it’s location based so you actually deploy your bots or whatever from who you actually are, so fight it over your own neighborhood. It’s kind of fun. That’s my pick! CHUCK: Oh, interesting. JOEL: Disclaimer: I did version 1.0 over a year ago, and I was only involved about a month [chuckles]. CHUCK: [Laughs] JAIM: Oh, okay. I did know you. It does one of the stuff on that. JOEL: [Laughs] No, it’s fine. I actually am getting more involved on themes and more help, so I’ve been coming back and kind of help them clean up the cloud base in the last couple of months. They did some great things, as someone would plot, keep an eye on it. CHUCK: So all the good parts, that was Joel. JOEL: That was him. CHUCK: Is that what you’re saying? JOEL: Yeah [laughs]. It’s getting better. CHUCK: Awesome. Alright, I just got back from RubyConf, and I had a pretty positive experience; tried some new things. So the first thing that I’m going to pick is actually “AirBnB”. By the time I actually got around to booking the hotel, all the hotel rooms in the conference block were taken. So rooms were going for like $550 a night, and I just wasn’t willing to pay that. So I went on AirBnB and I found an apartment that I could stay at, and that cost me $500 for the whole trip. It was an alright apartment. It was obviously much cheaper than a hotel room, but it had like a window unit, airconditioner, and stuff so it was cooler than outside, but not as cool as I would have liked. But other than that, it was nice. It was nice being in kind of in my own little space, and it was bigger than a hotel room; lots of homey features like that. Overall, I was pretty happy with it. So AirBnB is one pick. The other pick is “Delta Airlines”. Now, they have a hub here in Salt Lake City so I usually travel on them. I’ve traveled some of the discount airlines that service Salt Lake City, but I have never ever, ever been happy with their service, and Delta has always been pretty good, so I’m going to pick them, too. Joel, what are your picks? JOEL: Firs one is actually a To-Do list application, but it has a lot of start. It’s modeled after Portal’s AI, it’s called “CARROT”, MeetCarrot.com. It’s fun. It harasses you on a daily basis to get stuff done, and it makes me laugh every time I use it. So, it’s great. Another pick would be for the small amount of consulting work that I do. I use “Harvest” which is a great program to keep track of time, then automate your invoicing, and keeping track of pings and everything. I tried a lot of different solutions, but Harvest is kind of the best especially when it comes to working with lots of clients and you try to build by the minute. So it’s – I know it’s GetHarvest.com. CHUCK: Yup, it is. JOEL: Those are my picks! JAIM: I’ve been using CARROT for a few months, and I like it. It keeps me on-task by insulting me constantly. CHUCK: Yeah, I use Harvest for my consultancy and stuff. I’ve been pretty happy with them. So yeah, I guess that’s it! Thanks for coming, Joel. We really appreciate you taking the time! JOEL: Thanks for having me! CHUCK: Alright! Well, we’ll wrap up the show. We’ll catch you all next week!