Towards Reactive Server Apps: a new hybrid web programming model pioneered by Blazor

Summary: Microsoft recently announced Razor Components (formerly Server-Side Blazor) will be shipping in .NET Core 3. Razor Components offer a new kind of programming model for the web, a blend of SPA and classic POST + Redirect + GET apps.

image

Reactive Server Apps: A fully-reactive web stack, where changes in the UI are automatically pushed down to the database, and changes in the database are automatically pushed up through to the DOM.

Imagine writing a web app that has virtually zero JavaScript, doesn’t need page reloads, and changes to the database are automatically and instantly reflected in the UI.

The Blazor project moves towards this ideal via its Razor Components (formerly Server-Side Blazor) programming model, what we might call the Reactive Server App model.

Today, most web apps fall into 2 categories:

  1. Classic POST + Redirect + GET (PRGs)
  2. Single Page Apps (SPAs)

POST+Redirect+GET is where you type in some data to a web page, hit submit button (POST), after a few seconds, a new page loads (Redirect) with the updated data (GET). You might call this classic web development.

Ordering tickets online is typically this kind of web app.

Single Page Apps (SPAs) are the thick-client model, only in the browser with JavaScript. You type in a URL, and the app loads. After that, everything seems to happen without page reloads: navigation, saving your data, loading data. This is because it’s all asynchronous; navigation and saving and UI interaction is done via asynchronous HTTP calls in the background.

Gmail is a Single Page App.

Blazor, the experimental framework intended to run C# in the browser via the new Web Assembly standard, is introducing a new hybrid model of web programming, what we might call Reactive Server Apps.

In v0.5, Blazor introduced Server-Side Blazor Apps. Where Blazor is Web Assembly runtime executing C# in the browser – a variant on SPA thick clients – Server-Side Blazor is a new programming model, where state lives on the server but is asynchronously pushed to the browser via web sockets.

Blazor went to v0.6 yesterday with a big announcement: Server-Side Blazor apps would be shipping separately from Blazor Web Assembly, shipping earlier (in .NET Core 3, due early 2019), and would be getting a new name: Razor Components.

How does this new web programming model work?

Like classic POST + Redirect + GET (PRG) apps, you writer server-side code (in our case, C# and .cshtml files). But unlike classic PRG apps, all the state is automatically shared between the server and the browser via web sockets.

So your app logic and state all exists on the server, but it’s automatically transferred to the browser. You click Save, and it feels like a SPA: things appear to happen instantaneously without POST + Redirect + GET reloads.

And because there’s a live connection between server and browser, apps no longer need to fetch data to display a page; the data can be pushed down to the browser in real time. This is why I call this hybrid model Reactive Server Apps.

If coupled with a database that supports reactive push notifications (e.g. RavenDB), this programming model may gain traction. I envision future web frameworks where the whole stack is reactive:

  1. Web app is reactive, pushing changes from the JavaScript data model to the DOM.
  2. Web server is reactive, pushing changes from the the server’s data model down to the browser.
  3. Database is reactive, pushing changes from the database to the web server.

Frameworks like React and Angular already do part #1: changes in the JavaScript model are automatically reflected in the DOM.

But these frameworks don’t do #2; instead they must poll the server for the latest data. And to do that, the server must poll the database.

Razor Components gives us step 2: the data model and state on the server is automatically pushed down to the browser. When the state or data changes, your server-side web app signals the browser and the corresponding browser components get asynchronously updated.

All that’s remaining is #3: a database that pushes changes to your web server. It just so happens we have such databases; I’m partial to RavenDB.

Imagine a server-side framework that has per-page state: here is the live data for /dashboard. As you’re looking at the page in your browser, changes to the data will appear automatically, because the database pushes changes to the server app, which pushes changes to the browser, which pushes changes to the DOM. The UI updates instantly and the developer didn’t have to do anything for that to happen.

Such a programming model has tangible improvements over both SPAs and PRGs.

Unlike SPAs, Reactive Server Apps would load fast because it’s essentially still a thin-client, no giant JS libraries or runtimes to pull in.

Reactive Server Apps would also be live: programmers don’t have to refetch data from the server to update the UI, and users experience immediate UI updates as the data changes; even if someone else changed the data they’re looking at.

Whereas SPAs rely heavily on Javascript, Reactive Server Apps will benefit from having more powerful programming languages to do the heavy lifting of development, easing development further.

And unlike PRGs, Reactive Web Apps don’t need page refreshes; the same benefit SPAs give. They also are “live” – changes to the database will flow to the DOM without having to re-query the database.

Downsides?

At least one downside is scalability; each app must have an open socket connection to the server. How well does this scale? This StackOverflow question seems to suggest it scales into the hundreds of thousands of concurrent users before having to add more servers.

Whether this model takes off or not is yet to be seen, but the idea is innovative. As a web developer, I’m excited to experiment with it.

I built a PWA and published it in 3 app stores. Here’s what I learned.

Summary: Turning a web app into a Progressive Web App (PWA) and submitting it to 3 app stores requires about a month of work, a few hundred dollars, and lots of red tape.

I recently published Chavah Messianic Radio, a Pandora-like music player, as a Progressive Web App and submitted it to the 3 app stores (Google Play, iOS App Store, Windows Store).

image

image

image

The process was both painful and enlightening. Here’s what I learned.

Why?

First, you might wonder, “Why even put your app in the app stores? Just live on the opened web!”

The answer, in a nutshell, is because that’s where the users are. We’ve trained a generation of users to find apps in proprietary app stores, not on the free and open web.

For my web app, there were 2 big reasons to get in the app store:

  1. User demand
  2. Web app restrictions by Apple hostile mobile platforms

User demand: My users have been asking me for years, “Is there an app for Chavah? I don’t see it in the store.”

They ask that because we’ve trained users to look for apps in proprietary app stores.

My response to my users has up until now been,

“Aww, you don’t need an app – just go to the website on your phone! It works!”

But I was kind of lying.

Real web apps only kinda-sorta work on mobile. Which brings me to the 2nd reason: web app restrictions by Apple hostile mobile platforms.

Mobile platform vendors, like Apple, are totally cool with apps that use your phone to its fullest. Access your location, play background audio, get your GPS coordinates, read all your contacts, play videos or audio without app interaction, read your email, intercept your typing, play more than one thing at a time, use your microphone and camera, access your pictures, and more.

Apple’s totally cool with that.

But only if you pay Apple $99/year for the privilege.

If you want to do any of those things in a regular old web app, well, goshdarnit, Apple won’t just deny you these things, it prevents you from even asking permission.

For my Pandora-like music player app, this horrible brokenness showed up in numerous ways.

From minor things like “iOS Safari won’t let you play audio without first interacting with the page” to major, show-stopping things like, “iOS Safari won’t let you play the next song if your app is in the background or if your screen is off.”

Oh, plus weird visual anomalies like typing in a textbox and seeing your text appear elsewhere on screen.

So, to make my HTML5 music app actually functional and working on people’s mobile devices, it was necessary to turn my PWA into an app in app store.

Barriers to entry

In the ideal world, publishing your web app to the app stores would look like this:


Your Web/Cloud Host or CI Provider

You’ve published a Progressive Web App. Publish to app stores?

☑ iOS App Store
☑ Google Play
☑ Windows Store


(Or alternately, as Microsoft is experimenting with, your PWA will just automatically appear in the app store as Bing crawls it.)

But alas, we don’t live in this ideal world. Instead, we have to deal with all kinds of proprietary native BS to get our web apps in the stores.

Each of the app stores has a barrier to entry: how difficult it is to take an existing web app and it in the app store.

I list some of the barriers below.

Cost


  • Apple: $99/year to have your app listed in the iOS app store.
     
  • Google: One-time $25 fee to list your app in the Google Play Store.
     
  • Microsoft: Free!

Don’t make me pay you to make my app available to your users. My app enriches your platform. Without good apps, your platform will be abandoned.

Apple used to understand this. When it first introduced the iPhone, Steve Jobs was adamant that HTML5 was the future and that apps will simply just be web apps. There was no native iPhone SDK for 3rd parties. Apple has since abandoned this vision.

Google asked for a token $25 one-time fee. Probably to avoid spammers and decrease truly junk apps from entering the store.

Microsoft seems determined to just increase the total number of apps in their app store, regardless of quality.

Winner: Microsoft. It’s hard to beat free.

Adding native capabilities

In an ideal world, I wouldn’t have to write a single extra line of code for my web app to integrate into the OS. Or, as Steve Jobs said back in 2007,

“The full Safari engine is inside of iPhone. And so, you can write amazing Web 2.0 and Ajax apps that look exactly and behave exactly like apps on the iPhone. And these apps can integrate perfectly with iPhone services. They can make a call, they can send an email, they can look up a location on Google Maps.”

-Steve Jobs, 2007

For me, that means my web app plays background audio using standard HTML5 audio; that works just fine on all OSes.

My web app declares what audio is playing, and the OSes pick up on that, show currently playing song info on the lock screen.

My app controls audio using standard HTML5 audio API; the OS picks up on that and provides play/pause/next/volume/trackbar controls on the lock screen.

But sadly, we don’t live in this ideal world. All the things listed above don’t actually work out of the box on all 3 platforms.

My web app needs to play audio in the background. And load URLs from my CDN. Sounds reasonable, right? And bonus, how about showing currently playing song info on the lock screen? And controlling the audio (play/pause/next, etc.) from the lock screen? How hard is this?

Three very different approaches taken here:


  • Apple: We don’t give web apps a way to declare such capabilities; you’ll need to write a native wrapper (e.g. with Cordova) to interact with the OS.
     
  • Google: Web FTW! Let’s create a new web standard that shows audio & controls from the lock screen. Background audio? Sure, go ahead!
     
  • Microsoft: We’ll inject our proprietary API, window.Windows.*, into your JavaScript global namespace and you can use that to do the things you want to do.

Going into more details here for each store:

For iOS app store, does your web app need to play background audio? Use a Cordova plugin. Need to show currently playing song on the lock screen? Use a Cordova plugin. Need to control the currently playing song from the lock screen? Use a Cordova plugin. You get the idea. Basically, Cordova tricks Apple into thinking you’re a native app. And since you’re not a yucky web app, Apple lets you do all the things native apps can do. You just need native tricks – Cordova plugins – to let you do it.

For Google Play, it’s nice that I can just write JS code to make this work; no Cordova plugins required here. Of course, that JS won’t work anywhere except Chrome on Android…but hey, maybe one day (in an ideal world!) all the mobile browsers will implement these web APIs…and the world will live as one. I’m almost ready to bust out some John Lennon hippie utopia tunes.

For Windows Store, do you want to play background audio? Sorry! That is, unless you declare your intentions in our proprietary capabilities manifest file (easy) AND you implement this proprietary media interface using window.Windows.SystemMediaTransportControls (not so easy). Otherwise we’ll mute you when your app goes to the background.

Winner: Google. I want to be able to just write JavaScript, and let the OS pick up cues from my app.

Runner-up: Windows. I can still write plain old JavaScript, but I need to talk to a proprietary Windows JS API that was injected into my process when running on Windows. Not terrible.

Loser: Apple. They don’t care about web apps. Actually, it’s worse than that. It feels like they are actually hostile to web apps. iOS Safari is the new Internet Explorer 6. It has lagged behind in nearly every web standard, especially around Progressive Web Apps. This is probably for business reasons: web apps disrupt their $99/year + 33% in-app purchases racket. So to make my web app work on their platform, I have to basically pretend I’m a native app.

App Store Registration

Submitting your PWA to the app store requires registration, business verification, and more red tape. Here’s how the 3 app stores fared:


  • Apple: You must prove that you’re a legal, registered business. This verification isn’t done by us – but by a 3rd party, which may or may not know about your business.
     
  • Google: You want your app in our store? Cool by us.
     
  • Microsoft: You want your app in our store? Cool by us.

The biggest pain point for me was getting verified as a legal business by Apple.

First, I went to the site and registered for Apple’s Developer Program. I filled out my name and company information. (Aside: I guess Apple won’t let you submit an app unless you have a registered, legal company?)

I click next.

“The information you entered did not match your D&B profile.”

My…what?

A bit of Googling showed that “D&B profile” is Dun and Bradstreet. I’ve never heard of this group before, but I find out that Apple is using them to verify you legal corporation details.

And apparently, my D&B profile didn’t match what I put in my Apple Dev registration.

I google some more and find the Apple dev forums littered with similar posts. Nobody had a good answer.

I contact Apple Dev support. 24 hours later, I’m contacted by email saying that I should contact D&B.

image

I decide to contact them…but Apple says it will take up to a few days for them to respond.

At this point, I’m thinking of abandoning the whole idea.

While waiting for D&B support to get back to me, I decide to go to the D&B site, verify my identity, and update my company information which, I assume, they had taken from government registration records.

Did I mention how sucky this is? I just want to list my existing web app in the store. Plz help.

I go to D&B to update my business profile. Surprise! They have a JavaScript bug in their validation logic that prevents me from updating my profile.

Thankfully, I’m a proficient developer. I click put a breakpoint in their JavaScript, click submit, change the isValid flag to true, and voila! I’ve updated my D&B profile.

Back to Apple Dev –> let’s try this again. Register my company…

“Error: The information you entered did not match your D&B profile.”

AREYOUFREAKINKIDDINGME.

Talk to Apple again. “Oh, it may take 24-48 hours for the updated D&B information to get into our system.”

You know, because digital information can take 2 days to travel from server A to server B. Sigh.

Two days later, I try to register…finally it works! Now I’m in the Apple Developer program and can submit apps for review.

Winner: Google and Microsoft; both took all of 5 minutes to register.

Loser: The Apple Developer registration was slow and painful. It took about a week to actually get registered with their developer program. It required me contacting support from 2 different freaking companies. And it required me to runtime debug the JavaScript code on a 3rd party website just so that I can get past their buggy client-side validation, so that my info will flow to Apple, so that I can submit my app to the store. Wow, just…wow.

If there is any saving grace here for Apple, it’s that they have a 501c3 non-profit program, where non-profits can have their $99 annual fee waived. I took advantage of that. And perhaps this extra step complicated matters.

App Packaging, Building, Submitting

Once you have a web app, you have to run some magic on it to turn it into something you can submit for App Store review.


  • Apple: First, buy a Mac; you can’t build an iOS app without a Mac. Install XCode and these build tools and frameworks, acquire a certificate from our developer program, create a profile on a separate website called iTunes Connect, link it up with the certificate you generated on the Apple Dev center, then submit using XCode. Easy as one, two, three…thirty-seven…
     
  • Google: Download Android Studio, generate a security certificate through it, then package it using the Studio. Upload the package to Android Developer website.
     
  • Microsoft: Generate an .appx package using these free command line tools, or Visual Studio. Upload to the Microsoft Dev Center website.

The good news is, there’s a free tool to do the magic of turning your web app into app packages. That awesome free tool is called PWABuilder. It analyzes a URL, tells you what you need to do (e.g. maybe add some home screen icons to your PWA web manifest). And in a 3 step wizard, it lets you download packages that contain all the magic:

  • For Windows, it actually generates the .appx package. You can literally take that and submit it on the Windows Dev Center site.
     
  • For Google, it generates a wrapper Java app that contains your PWA web app. From Android Studio, you build this project, which generates the Android package that can be uploaded to the Android Dev Center site.
     
  • For Apple, it generates an XCode project which can be built with XCode. Which requires a Mac.

Once again, Apple was the most painful of all of these. I don’t have a Mac. But you cannot build the XCode project for your PWA without a Mac.

I don’t want to pay several thousand dollars to publish my free app in Apple’s app store. I don’t want to pay for the privilege of enriching Apple’s iOS platform.

Thankfully, MacInCloud costs about $25/month, and they give you a Mac machine with XCode already installed. You can remote into it using Windows Remote Desktop, or even via a web interface.

It wasn’t enough to just build the XCode project and submit. I had to generate a security certificate on the Apple Developer site, then create a new app profile in a separate site, iTunes Connect, where you actually submit the package.

And that wasn’t all: since Apple is hostile to web apps, I had to install some special frameworks and add Cordova plugins that allow my app to do things like to play audio in the background, add the current song to the lock screen, control the song volume and play status from the lock screen, and more.

This took at least a week of finagling to get my app into a working state before I could submit it to the app store.

Winner: Microsoft. Imagine this: you can go to a website that generates an app package for your web app. And if that’s not your thing, you can download command line tools that will do the job. GUI person? The free Visual Studio will work.

Runner-up: Google. Requires Android Studio, but it’s free, runs everyone, and was simple enough.

Loser: Apple. I shouldn’t have to buy a proprietary computer – a several thousand dollar Mac – in order to build my app. The Apple Dev Center –> iTunes Connect tangling seems like an out-of-touch manager’s attempt to push iTunes onto developers. It should simply be part of Apple Developer Center site.

App Testing

Once you finally did all the magic incantations to turn your existing web app into a mobile app package, you probably want to send it out to testers before releasing your app on the unwashed masses.


  • Apple: For testing, you have your testers install Test Flight on their iOS device. Then you add the tester’s email in iTunes Connect. The tester will get a notification and can test your app before it’s available in the app store.
     
  • Google: In Android Dev Center, you add email addresses of testers. Once added, they can see your alpha/beta version in the App Store.
     
  • Microsoft: I didn’t actually use this, so I won’t comment on it.

Winner: Toss up. Apple’s Test Flight app is simple and streamlined. You can control alpha/beta expiration simply on the admin side. Google wasn’t far behind; it was quite painless, not even requiring a separate app.

App Review

Once your app is ready for prime time, you submit your app for review. The review is done using both a programmatic checklist (e.g. do you have a launch icon?) and by real people (“your app is a clone of X, we reject it”)


  • Apple: Prior to submission, XCode warns you about potential problems during build. The human app review takes about 24-48 hours.
     
  • Google: Anybody home? Android Studio didn’t tell me about any potential problems, and my app was approved within minutes of submission. I don’t think a real human being looked at my app.
     
  • Microsoft: Upon submission, a fast programmatic review caught an issue pertaining to wrong icon formats. After passing, a human reviewed my app within 4 days.


Winner: Apple.

Sure, as a developer, I like the fact that my app was instantly in the Google Play store. But that’s only because, I suspect, it wasn’t actually reviewed by a human.

Apple had the quickest turnaround time for actual human review. Updates also passed review within 24 hours.

Microsoft was hit or miss here. The initial review took 3 or 4 days. An later update took 24 hours. Then another update, where I added XBox platform, took another 3-4 days.

Conclusion

It’s painful, and costs money, to take an existing PWA and get them functional on mobile platforms and listed in the App Store.

Winner: Google. They made it the easiest to get into the app store. The made it the easiest to integrate into the native platform, by attempting to standardize web APIs that OS platforms can pick up on (hello, lovely navigator.mediaSession)

Runner-up: Microsoft. They made it the easiest to sprinkle your web app with magic, turning it into a package that can be submitted to their store. (Can be done for free using the PWABuilder site!) Integrating with their platform means using the auto-injected window.Windows.* JavaScript namespace. Not bad.

Loser: Apple. Don’t require me to buy a Mac to build an iOS app. Don’t force me to use native wrappers to integrate with your platform. Don’t require me to screw around with security certificates; let your build tools make them for me, and store them automatically in my Dev Center account. Don’t make me use 2 different sites: Apple Dev Center and iTunes Connect.

Final thoughts: The web always wins. It defeated Flash. It killed Silverlight. It destroyed native apps on desktop. The browser is the rich client platform. The OS is merely a browser-launcher and hardware-communicator.

The web will win, too, on mobile. Developers don’t want to build 3 separate apps for the major platforms. Companies don’t want to pay for development of 3 apps.

The answer to all this is the web. We can build rich web apps – Progressive Web Apps – and package them for all the app stores.

Apple in particular has a perverse incentive to stop the progress of the web. It’s the same incentive that Microsoft had in the late ‘90s and early 2000s: it wants to be the platform for good apps. PWAs undermine that; they run everywhere.

My software wisdom is this: PWAs will eventually win and overtake native mobile apps. In 5-10 years, native iOS apps will be as common as Win32 C apps. Apple will go kicking and screaming, keeping iOS Safari behind the curve, blocking PWA progress where they can. (Even their recent “support” for PWAs in iOS Safari 11.1 actually cripple PWAs.)

My suggestion to mobile app platforms is embrace the inevitable and either automatically add quality PWAs to your app store, or allow developers to easily (e.g. free, and with 3 clicks or less) submit a PWA to your store.

Readers, I hope this has been helpful glance at PWAs in App Stores in 2018.

Have you submitted a PWA to an app store? I’d love to hear your experience in the comments.

Enabling TypeScript 2.0 strict null checks in a Visual Studio project

TypeScript 2.0 beta was released today. Among the big list of 2.0 awesomeness, the headline feature is non-null types.

Non-null types is currently opt-in: you pass a –strictNullChecks flag to the compiler to enable non-null types.

Enabling this feature in a Visual Studio project wasn’t obvious to me. This post shows how to do that.

Once you’ve downloaded the TypeScript 2.0 beta for VS 2015, open your project in Visual Studio. It will prompt you to upgrade the project to the new TypeScript tooling.

Once you’ve done this, open your .csproj in a text editor. Scroll down and find the TypeScript property group:

You’ll want to add the line:

<TypeScriptStrictNullChecks>True</TypeScriptStrictNullChecks>

inside the property group, as shown above.

Save the .csproj, reload it in Visual Studio, and the feature will be enabled.

Bonus: this is also how you’d enable the new 2.0 compiler flags, noUnusedParameters and noUnusedLocals:

<TypeScriptNoUnusedParameters>True</TypeScriptNoUnusedParameters>
<TypeScriptNoUnusedLocals>True</TypeScriptNoUnusedLocals>

“Yet Another JS Framework” isn’t so bad after all

Summary: The web is evolving. Fast. The default reaction is framework fatigue: “Another JS framework to learn? Ugh!” But there’s a hidden upside to the rapid evolution in front-end web dev, and it benefits your happiness, your mind, and your wallet.

We’re seeing rapid evolution in the web development space. Not only is JavaScript and HTML evolving for today’s needs, but numerous frameworks are popping up to leverage the new bits and help you build better web apps.

This fast-paced evolution can be difficult for web developers. There’s a never-ending stream of new things to learn, and that gets overwhelming.

image

My experience In the last 2 years testifies to this: I’ve built web apps with plain old jQuery, then Knockout, then Durandal, then Angular. And with today’s announcement, I’ll probably write my next one with Aurelia. That’s 5 different libraries/frameworks in just 2 years! Web devleopment is a beautiful soup.

But this does lead to developer framework fatigue. On Hacker News today, for instance,

image

Having to learn new stuff all the time and trying to keep up is fatiguing and can be overwhelming.

But there’s another side of this to consider. Rapid evolution leads to better ideas, keeps your mind fresh and your job more interesting. It gives developers an opportunity to stand out with new skills and make more money.

Better ideas rise to the top

Make no mistake: web development has improved by leaps and bounds in the last decade. Rapid library and framework evolution are to thank.

(If a crotchety developer pining away about “the good ol’ days” tells you otherwise, he’s not telling the truth. Those old days were not good. They were littered with browser incompatibilities, unmaintainable apps, no concern for architecture or testing. It was truly the wild west of coding, and it sucked.)

When I started web development, front-end JavaScript meant adding a single giant JavaScript file where you’d do your DOM manipulation and event handlers. The DOM manipulation was usually using browser-specific APIs, and the handlers were hooked up in the global namespace.

This was like the Visual Basic era: put everything into a single file, maybe even a single giant function! We didn’t care about maintainability, testing, separation of concerns.

It was a simpler time, yes, but it was also the time when web development and JavaScript received ugly reputations that persisted until only recently.

Back in the bad ol’ days, web apps themselves generally sucked, too, as a reflection of the code underneath. Great web apps like Gmail, DropBox, Evernote, Pandora…these didn’t exist because building web apps was hard. (Pandora existed…but as a plugin-based Flash app back then!)

Since then, things have vastly improved. How?

Instead of a spaghetti mess of $(“#someVal”).parent().adjacent(“<div>”+ someVar + “</div>”), we’re doing nice and clean logic on variables. Through the magic of data-binding, those variables will be displayed in the UI automatically. The code is modular: simple HTML pages data-bind to controllers containing logic. Controllers wire up to services to load data from the server. The code is arranged in modules with single responsibility, well-defined application architecture, easy to test, easy to modify and extend. This results in better web apps with fewer bugs.

Intellectually stimulating

I actually like the fact that I must keep learning.

It’s true; web development can be overwhelming with all the new technologies and JavaScript frameworks released almost monthly. (If you’re overwhelmed, remember: you don’t have to learn them all.)

But at the same time, I like learning. It keeps my mind fresh. It keeps my job interesting. I like building things, and learning new and better ways to build things is intellectually stimulating.

Consider the alternative. In a space with slow evolution, such as mainframes, there’d be practically no moving technology forward; we’d be doing the same thing for decades, over and over again. See something that sucks about your dev stack? Too bad, you can’t change it.

As a developer and entrepreneur, I want to be doing new things, interesting things, moving the software craft forward. It is a little ego-stoking to think that moving software forward actually moves technology and civilization forward, even if in a small way.

I don’t believe there’s a world in which my skills go stagnant and I still enjoy building technology. It would get old, I would grow tired of my work, become demotivated. My job would become only a means to pay the bills.

I don’t want that career. I like a career that remains fresh, regularly infused with new knowledge. That new knowledge can then be used outside of the professional realm to build your own projects of value to you, personally. (I’ve done this with Chavah, MessianicChords, and EtzMitzvot.)

“Look ma, I’m worth a lot of money”

Knowledge is power. The more I consume, the more powerful (and valuable) I become. Therefore, I am actually grateful to be in a career that requires knowledge growth.

My last 2 gigs as an independent consultant were aided by my knowledge of a single JavaScript framework. And my hack-at-night job gives me money because I know another JavaScript framework. And my startup exists because of the new knowledge I’ve learned in the last two years.

Rapid evolution demands me to acquire more knowledge to remain relevant. I’d be hard-pressed to find work if I was still building web apps like it was 1998, or worse, building desktop apps like it was 1995. (Goodbye, CListViewCtrl. No tears.)

Summary

By continually learning, not only am I intellectually stimulated, I grow in professional value. Corporations pay a lot of money to the niche few who keep their skills relevant. They stand in awe and fork over the dollars by the truckload for the magical wizards who pound on the keyboard and produce revenue-generating, business-critical apps.

So yeah, Yet Another JS Framework was announced today. And I’m OK with that.

Show and tell: Beds and services for homeless youth

Check it out! Awesome success story here: my last project, ysnmn.org, is live! It’s a place for homeless youth to find beds and services in Minnesota. Also, we made it to the front page of Hacker News!

image

I’m a big fan of helping people, and I think this project really does that.

Imagine you’re a homeless 16 year old. You need food, you need a bed to sleep in. Somewhere to shower. Cheap clothing. Maybe you need medical care or counseling. Or help with parenting your child. Maybe you’re looking for a way to finish your education and get back on your feet and off the streets.

Enter ysnmn.org. You pull out your phone, go to the website. Instantly you can see shelters nearest you:

image

What buses to take from your current location to get to a drop-in center:

image

What food shelves are opened and what they serve:image

What phone numbers to call, where to get medical help:image

Where to find a warm bed for the night:
image

Or maybe you just need to be notified when a bed for a female 16 year old becomes available:
image

image

It was such a pleasure building this thing! Working with people who are actually doing good in the world through helping homeless youth is a big win. It’s so satisfying building something for goodness, rather than just for business. The most memorable moment on the project was testing the app with youth ambassadors; I remember one of the kids turning to me and saying, “I wish I had this when I was homeless.” Big smile right there on yours truly.

Technologically, I had a blast building the app as the lone developer! Smile This was a nice change from my previous gig. I had quit my consulting day job to take this project – working as independent consultant in the process – and being the lone developer, I was able to architect the app as I saw fit. We were required to use the Microsoft stack, seeing as how they funded development. But I was cool with that – C# is a great language, Azure is increasingly a solid platform, and TypeScript on the client was a pure joy. I would have preferred to use RavenDB over MS SQL, but aside from that, the freedom of architecture was quite liberating from previous projects working as a cog in a bigger corporate team.

It was my first stab at using AngularJS. I had built Knockout, Durandal, and classic MVC apps in the past, but this was my first stab with Angular. I liked it enough that I’ve adopted it for most of my side projects.

Many shout outs to Microsoft and their Azure group, and particularly Adam Grocholski who not only facilitated this, but also helped us secure some additional credits and put out a few fires during development, and his wife Ann Marie Grocholski who heads up one of the local youth shelters. You guys rock!

A big thank you to the Target Foundation, who helped fund this project with a generous grant.

A big shout out to DevJam in Minneapolis. They approached me to build this project, paid me a nice sum, provide great working environment, and are generally very cool people. I had a great time working with them. Special thanks to Matt Bjornson who worked with me and the shelters the whole time, came up with some great mockups for the UI, and saw this thing to completion. Special thanks to Susan Greve for recruiting me onto this project. So glad I took it!

A shout out to Twilio, particularly Kevin Whinnery, who helped us get started with Twilio and gave us a bunch of free credits. We are putting them to good use, using them to text homeless youth of available beds. Winning!

A shout out to SendGrid and their social network people. I showed them an early prototype of this app, told them I intended to use SendGrid for emailing homeless youth when beds became available. Their reaction? A big account with lots of free credits, plus a t-shirt for yours truly!

This was a great project. I hope to work on more like it.

RavenDB MVC starter kit

Looking to get started building ASP.NET MVC apps with RavenDB? Check out RavenDB.ModernMvcStarterKit. It’s an MVC template that lets users register on your site, verify their registration via email, and allows them to opt-in to two-factor authentication via email or SMS.

Modern websites need a robust identity system, where users can login, confirm their registration via email, and optionally enable two-factor authentication.

Doing this with the .NET + RavenDB stack has had some friction:

  • The default MVC template is wired to work with Entity Framework, not RavenDB.
  • Making the default template work with RavenDB is non-trivial, and requires implementing a RavenDB-specific identity provider.
  • There is no publicly-available Identity provider for RavenDB. Or rather, there are 2 available, but they work against old versions of Identity Framework: tugberkugurlu’s AspNet.Identity.RavenDB and ILM’s RavenDB.AspNet.Identity.
  • The basic MVC template from Visual Studio doesn’t implement email service or SMS service, nor UI for this or Two-Factor Authentication, leaving it up to you to do all the heavy lifting.

To remove this friction, I created RavenDB.ModernMvcStarterKit.

In a nutshell, it’s an MVC sample project that uses RavenDB as the backend, supports user registration and confirmation via email (SendGrid), and optionally supports Two-Factor Authentication via email (SendGrid) or SMS (Twilio).

This project provides:

  • A RavenDB-backed identity provider, updated to work with the latest MS Identity Framework (2.1 at the time of this writing). This is a fork of ILM’s RavenDB.AspNet.Identity provider, updated to work with email and SMS authentication and the latest Identity bits. I’ve submitted a pull request with my changes to merge back into that repo.
     
     
  • A registration confirmation system: when a user registers, he is required to confirm his registration via email. The email is sent via your developer SendGrid account. Upon registering, the user is presented with the following screen:
    The user will receive an email with a confirmation link. Following the link, he’ll be taken to this page:
     
     
  • Optional two-factor authentication: Users can optionally go to their profile page and add a phone number and enable Two-Factor Authentication:
    When the user enters his phone number, we send an SMS verification code via your developer Twilio account:
    With Two-Factor Authentication enabled, when the user goes to sign-in next time, he will first login as usual, and then be prompted to enter the 2nd form of identification, either email or SMS:
    The user will receive a verfication code via SMS or email and be redirected to the verification page:
    Upon entering the verification code, they’ll be able to sign in.

Bottom line: this is a nice way to build a modern MVC app with RavenDB. It’s all about removing friction. Smile Check out the project on GitHub.

Enjoy!

Lap Around HTML5 RAvenDB Studio

Last month, I was honored to give a talk at the very first (!) RavenDB conference, the first of many, as I believe Raven is poised for greatness as it gains more exposure and spreads its wings outside the .NET niche.

My talk was on the new Raven Studio – built from the ground-up using modern web technologies, HTML5, LESS, TypeScript – oh yes!

[youtube https://www.youtube.com/watch?v=wlbAj-qEuV4]

I hope you guys enjoy the new RavenDB Studio! It’s now available as unstable 3.0 build. Browse the code (or heck, send me pull requests) over at the GitHub repo.

Charlatan developers and the Blunt truth

Summary: A rant on political correctness in the programming world.

Ayende, developer of RavenDB and author of popular ayende.com programming blog, has been blogging the ugly, cringeworthy interviews resulting from his company’s recent hiring round.

Some developers couldn’t sort a list of strings. Others didn’t know what framework they were developing in. Aptly summing the lot, one developer with 6 years experience and a CV mentioning multi-threading experience said, "I only know BackgroundWorker."

The short of it is, we have a lot of charlatans in our industry. (See Why can’t programmers…program?)

But the comments to these posts tell a different story: bleeding hearts sympathizing with the interviewee and chiding Ayende for blogging about the bad interview.

Kelly Somers (@kellabyte), big data extraordinaire, complained:

"As an employer, I don’t think posts like this are a professional way to behave. Although they are anonymous, there is a human trying to make a living being humiliated here and I can only imagine how they might feel after having a bad interview only to then read it up on the internet in the public eye."

Patrick Smacchia, creator of NDepend, chimed in agreement,

"I am with Kelly here…You’d better mention only positive things that happen in the interview room."

“Only the positive things.” ಠ_ಠ

I have much respect for both Kelly and Patrick and the things they’ve built. However, what’s more important, a person’s hurt feelings or telling the truth?

If I was that interviewee, I’d feel bad, sure, but I’d also come to appreciate the truthful feedback. I’d want to know that I’m failing to use loops. Had I interviewed at DataStax or NDepend, I’d probably be calling them repeatedly, receiving uncertain answers about whether I’m hired or not, everyone beating around the bush and no one telling me just why it is that my calls aren’t being returned.

Contrast with Ayende’s approach, I’d just know straight up my current skills are lacking, and that I shouldn’t be advertising my many years of developer expertise if I can’t work with for loops.

When I visited Israel the other year, people were blunt, to-the-point, and honest. It was jarring, but good. Having lived in the US for my entire life, it’s been drilled into me that you never say what’s really on your mind. Criticism should be withheld for sake of politeness. If it will offend someone, nobody really tells the truth here.

Ayende’s honesty is wonderful for that reason. Yes, even if it were my code being criticized, I’d appreciate the honesty. Wouldn’t you?

I suspect this blunt honesty is foreign to most folks from the US, and is the reason some people are chiming in with bleeding hearts. They think "positive only" is better. “Think of the human being searching for a job!” But this sort of thinking is not really thinking at all, but rather emoting, thinking with your emotions rather than intellect. Compassion to the point of untruthfulness is not actually helpful to human beings.

I’d rather know the truth, even if it hurts. Wouldn’t you?

/rant

My beautiful web development soup

Summary: Web development is chaotic, overwhelming, and beautiful. How many technologies does it take to build something useful on the web? You might be surprised. Advice for developers getting into web development.


3654636770_3b1a5d470bThis week I was working on my open source startup project, BitShuva Radio. I acquired a new client this week, but this client was unique because he’s a Java developer and wanted to know how to edit the open source code for his new radio station.

As I typed out my answer, it dawned on me that the sheer number of  technologies utilized in building a useful web application is staggering. That they actually work together is a testament of the beautiful soup that is web development today.

How many technologies does it take to build something useful on the web? Here’s my answer:

  • I use TypeScript, rather than plain JavaScript, for all client-side code. A superset of JavaScript with optional types, TypeScript is a well-structured, typed language with excellent tooling. I use it for practical reasons: as the app grew in complexity, the need for a more structured language became apparent. I ported the JS to TypeScript last month, and I’m quite pleased with the results.
  • KnockoutJS is used heavily absolutely relied on. BitShuva is a dynamic single page application, meaning we’re doing lots of dynamic updates to the UI in real time. Knockout lets us make dynamic UIs very easily without resorting to tons of DOM manipulation code. Instead of DOM manipulation, Knockout lets you change your objects, and your UI automatically updates. For example, songsOnScreen.push(new Song()) will automatically show the song in the UI, no jQuery DOM manipulation required. 
  • I use knockout.postbox for decoupled pub/sub communication between client-side code components. For example, when I hit the play button, the click handler doesn’t need to know about HTML5 audio or any infrastructure concerns. Instead, it calls ko.postbox.publish(“Play”). Whoever is in charge of playing audio simply calls ko.postbox.subscribe(“Play”, …). This way, the click handler and the HTML5 audio component don’t need to know about each other. Nice and clean. Bonus: it works with Knockout observables, so you can say currentSong.publishOn(“SongChanged”) to automatically publish messages on a particular topic.
  • UbaPlayer is used for playing HTML5 audio with a Flash fallback. 
  • For styling, we use LESS, the CSS superset. We use LESS because CSS is redundant; LESS lets us use variables and functions for code reuse, but still compiles down to plain CSS.
  • jQuery is still there, but not as much as you might think. Because KnockoutJS handles the DOM <—> code interaction, jQuery is only needed in rare cases where we need special DOM manipulation (e.g. to fade in/out an element). We also use jQuery to do AJAX calls.
  • The site generally uses Twitter Bootstrap UI toolkit for styling and consistency across the UI.
  • We’re using Google Web Fonts for typography. 
  • For images like thumb-up/down, play button, etc. these are actually font characters, from the special Font Awesome web font. Because it’s just a font, they scale infinitely without losing fidelity, and you can change their appearance (size, color, etc.) using plain old CSS stylings. Icon fonts are awesome.
  • We use Asp.NET MVC Razor view engine to actually render our UI. Because of heavy reliance on KnockoutJS, Razor isn’t needed much; it’s mostly just plain HTML that is dynamically swapped in through AJAX and KnockoutJS.
  • The server code is written in C# 5 using Asp.NET MVC 4 + MVC Web API
  • ASP.NET Web Optimization Framework is used for minifying and bundling the scripts and CSS.
  • The database is RavenDB, a modern, 2nd generation document database, easily the best NoSQL database in the .NET world. I use it because it’s blazing fast and simple to just start dumping objects into.
  • ReactiveExtensions (RX)  is used on the C# server end to treat events as observable streams, which can then write LINQ queries on top of. We also use its sister project, Interactive Extensions, for some useful extension methods to IEnumerable objects (such as .Do and .ForEach).
  • To load up the whole shebang and edit the project top-to-bottom, I use Visual Studio 2012 + TypeScript tools plugin (for editing/compiling TypeScript) + Web Essentials (LESS compilation, better tooling for HTML5 web standards, and more).

Holy cow!

Here’s the final tally of languages, frameworks, and tools used:

  • 5 languages – C#, TypeScript, LESS, .cshtml Razor syntax, plus bits of plain JavaScript here and there
  • 5 UI JavaScript frameworks – KnockoutJS, jQuery, UbaPlayer, Knockout.postbox, Twitter Bootstrap.
  • 6 server-side .NET frameworks – RavenDB, MVC 4, MVC Web API, Reactive Extensions, Interactive Extensions, Web Optimization Framework.
  • 3 tools – Visual Studio 2012, TypeScript tools plugin, Web Essentials plugin.

The sheer number of tools required to really build something useful on the web is staggering. You get the sense of why desktop developers — who often work in just a single language + UI framework (e.g. C# + WPF) — are hesitant to embrace the web. It requires massive retooling.

And to be truthful, this is only my beautiful web soup. I chose C# + KnockoutJS + TypeScript + LESS, but other web devs might prefer Ruby + BackboneJS + CoffeeScript + SASS. Or something entirely different.

The great thing is, there is so much evolution going on in web development space today, and this produces unique and specialized toolsets that help move the web forward. For example, languages like CoffeeScript and TypeScript are influencing the future direction of JavaScript.

It’s a fun time to be in web development.

My advice for developers? Learn a few web technologies that pique your interest and cook up your own beautiful web soup.