Summary: It’s a great time to be a web developer. HTML5, KnockoutJS, MVC3, RavenDB, and opened web APIs and services like Google Docs, IfThisThenThat, and RSS are combined to build a beautiful, useful web app. The opened web is coming to fruition.
I recently released MessianicChords.com, an HTML 5 site for finding lyrics and guitar chords to Messianic Jewish religious music.
Using opened web APIs to build this thing proved effective. Do web developers know these APIs exist? They should; these opened web APIs and standards are powerful tools that will make your app shine.
The raison d’etra of MessianicChords is to provide a way to find lyrics and guitar chords to Messianic Jewish music. This sounds simple, but building it presented a number of challenges:
- Precise character formatting required. Guitar chords sheets require precise formatting; the G chord must be precisely above the “Let us exalt” line, for example. This makes automatic conversion to straight HTML unfeasible. This lack of formatting preservation has plagued existing guitar chord sites like ultimate-guitar.com.
- Proprietary file formats. Guitar chord sheets are often in .doc or .pdf formats due to precise formatting requirements, as well as for historical reasons. This is problematic, because I wanted the chord sheets to be viewable on all devices and platforms, especially tablets like iPad, seeing as how many artists and guitarists use tablets to view the chords while playing music.
- Searching. I wanted to be able to type a title, artist, or even a partial lyric, and have results come back instantly. Today, users have come to expect Google-like instant search results. In reality, it’s very difficult to make search work like Google: partial matches, incomplete search terms, search qualifiers (such as placing quotes around a group of words) — these all present additional technical challenges.
- Letting users upload content. While myself and a few contributors have created a large library of Messianic Jewish music chord sheets, approximately several hundred chord sheets, letting the community add chord sheets was important.
- Automated announcement of new chord sheets. When new chord sheets are added, we’d like to announce it to the world. Better yet, make it automated.
- An interactive, clean HTML5 app. The existing chords and lyrics sites are littered with ads, have a messy navigation system, and look as though they were designed in the 1990s. Can we do better?
Getting complex, proprietary documents onto the opened web
The formatting issue was the most important to tackle: how can we take complex documents of proprietary file formats and make them available on the opened web?
Enter Google Docs. With Google Docs, you can upload documents, including .doc and .pdf formats, and view them right in your web browser without having to install any special software or plugins. Google’s servers do the heavy lifting on the back end, and you with your web browser reap the fruit.
Additionally, Google provides a sharing model for documents on Google Docs: you can share the document with individuals (specified by email address), you can make the document semi-public by generating a link to the document, or you can make the document entirely public, discoverable by search engines and the like. For MessianicChords, I chose the latter: documents that are public, discoverable by search engines, with a public URL. In addition to sharing, docs can have multiple authors and collaborators. For MessianicChords, I made the chord sheets visible on the web, but read-only to the general public, editable only by myself and some trusted admins.
Finally, and most importantly, Google Docs comes with a powerful API to find documents, metadata, and grab URLs to them. For MessianicChords, having chose ASP.NET MVC3 on the server, there’s a free .NET wrapper for the Google Docs APIs, available as a NuGet package.
With Google Docs, it allowed me to preserve the special formatting of guitar chord sheets, while still making them available on the web. With the Google Docs API, I was able to use Google Docs as a kind of database for the chord sheets. It solved the proprietary file format problem not by lossy conversion, as so many other chord sites do, but by making them viewable via pure HTML/JS.
Seaching
Users have come to expect Google-like search capabilities: instant, partial search terms, close matches, special search terms such as words grouped in quotes, and so on.
With Google Docs as our pseudo-database for chord sheets, I get the best of the search world: Google’s own search algorithms via the Google Docs API. Send over a query with bless God should return all documents with the word bless and God in them; many results. However, if you search for that same phrase, but in quotes, “bless God”, you get only documents that contain that exact phrase.
Best of all, the performance of these queries is excellent: near instant, under a second.
Using Google Docs as a storage mechanism for MessianicChords results in a free, web-accessible database with built-in Google search functionality. Win.
User-generated content and automating the “tell the world about it” part
Since MessianicChords allows users to contribute their own chord sheets, wouldn’t it be nice if we could announce to the world new chord sheets as they’re added?
Sure, I could send out an email, but email blasts are so 1990.
And yeah, I could put an announcement on the Facebook fan page, but Facebook is ultimately a walled garden that will eventually pass away.
Keeping with our spirit of living on the opened web, can we do some sort of announcement of new content that lives on the opened web?
Enter RSS. This is a long-standing, UI-agnostic technology that keeps track of updates to blogs, news sites, or other content that is frequently updated. Boiled down, it’s just an XML file that describes the what, who, and when of updates in a standardized way. The result can be consumed in any RSS aggregator, such as Google Reader.
For MessianicChords, I wanted to publish an RSS feed of new chord sheets. This way, updates to the website are broadcast in a standardized way on the opened web, not limited to Facebook walled gardens, private email inboxes, or proprietary services like Twitter. (Though we can push to those services if we please. More on that later.)
So, the first problem: we need to persistently store the list of updated documents. I could probably have used Google Docs for this, too, but it felt right to have a real database here to keep track of purely application-level things. For this, I used RavenDB Embedded. RavenDB is an open source document database. It’s a great, free database for .NET development, and the embedded version can run inside web apps without any special machine configuration or installation. I plopped that sucker right into my web app, following the lead from the excellent MSDN article, Embedding RavenDB into an MVC3 app.
Within 5 minutes, I was up and running with RavenDB, storing the user’s uploaded chord sheets as records in Raven.
With the upload records in place, how do you generate an RSS feed from ASP.NET MVC3?
Turns out, it’s a piece of cake, thanks in part to Microsoft’s Windows Communication Foundation (WCF) APIs, in particular, the SyndicationFeedFormatter class.
We’re looking to have a URL where the updates feed resides. For this, you can create a custom MVC ActionResult that wraps up an RSS feed.
public class FeedResult : ActionResult
{
public Encoding ContentEncoding { get; set; }
public string ContentType { get; set; }
private readonly SyndicationFeedFormatter feed;
public SyndicationFeedFormatter Feed
{
get { return feed; }
}
public FeedResult(SyndicationFeedFormatter feed)
{
this.feed = feed;
}
public override void ExecuteResult(ControllerContext context) {
if (context == null)
{
throw new ArgumentNullException("context");
}
HttpResponseBase response = context.HttpContext.Response;
response.ContentType = !string.IsNullOrEmpty(ContentType) ? ContentType : "application/rss+xml";
if (ContentEncoding != null)
{
response.ContentEncoding = ContentEncoding;
}
if (feed != null)
{
using (var xmlWriter = new XmlTextWriter(response.Output))
{
xmlWriter.Formatting = Formatting.Indented;
feed.WriteTo(xmlWriter);
}
}
}
}
This gives us an ActionResult we can then return from an MVC3 action to render an RSS feed. From there, we return an instance of this class, populating it with the upload records stored in RavenDB. End result? See it in action: MessianicChords.com/Home/UploadsFeed. As you can see, each upload record shows up in the RSS feed, as populated from Raven.
Wooohooo! We now have an RSS feed for user-generated content.
Pushing RSS to proprietary services like Facebook and Twitter
Is RSS enough? How many people know what RSS is, let alone know how to subscribe to it? Sadly, the current state of the web and web browsers makes it awkward at best to subscribe to an RSS feed.
How can we get our nice, standards-compliant, opened web feed of updates to regular, non-techie people?
Enter IfThisThenThat. In a nutshell, this is a free service that puts the web to work for you. In more concrete terms, it lets you listen for actions on the web — a new blog post, an RSS feed updated, a Tweet posted, a Facebook image tagged, whatever you can imagine — and then do some other action in response to that trigger. If [this], then do [that]. Scott Hanselman has a great write-up on the technology.
For MessianicChords, I decided my “this“, that is, my trigger, would be “when the MessianicChords RSS feed is updated”. And my “that“, which is my reaction to the trigger, would be, “Post a new Tweet announcing the new chord sheet.” It looks like this:
I also set up a trigger to announce the new chord sheet to Facebook. Also, to direct message my main Twitter account with the news, so that myself or other admins will quickly send it through the approval process.
Bottom line: we used the If This Then That cloud service to consume our “user uploads” RSS feed, then automatically tweet the news of the new user-generated content and spread it around the social networks. Awesome!
Interactive, clean HTML 5
Last but certainly not least, an important part of all this is the user experience. I want to differentiate my site from other chord sites by having zero ads, and a clean, responsive UI.
Yes, when search results come in, I want a subtle fade-in effect.
Yes, I want nice prompt text that disappears as you enter the textbox.
Yes, I want an inline upload functionality that doesn’t navigate away from the page or your current task.
Yes, I want each search to have its own URL.
Yes, I want search results to appear as I type. But if I type too fast, it should wait a moment before trying to search.
Yes, I want the whole thing to feel smooth and elegant.
While these things can be accomplished “manually”, as it were, with jQuery and straight DOM manipulation, this whole thing would become rather unwieldy.
Enter KnockoutJS. Knockout is a client-side library that makes it easy to build interactive web apps. It accomplishes this through data-binding and the MVVM pattern that separates your UI from your logic. Instead of a mash of UI DOM manipulation and logic in your JS code, with Knockout, you can write apps where your JS code free from DOM manipulation. Your code is cleaner — focused only on logic — allowing you to focus on doing awesome things in your UI.
For MessianicChords, I used data-binding to show the results of the search. My HTML looks like this:
<div class="search-results-container" data-bind="foreach: searchResults>
<div>
<a data-bind="text: Name, attr: { href: DocumentUrl }" />
</div>
</div>
So simple! Our JS, then, creates an array called searchResults, populates it at runtime. When it gets populated, Knockout renders a template for every item in searchResults array. The search results come back with objects containing a Name, a DocumentURL, and those get rendered inside our search-results-container as shown. Awesome!
How about things like automatically searching when we type? And only searching if the typing has been idle for awhile? Knockout to the rescue again:
<input data-bind="value: searchText, valueUpdate: 'keydown'" />
And in our JavaScript:
var searchResults = ko.observableArray();
var searchText = ko.observable();
var searchTextThrottled = ko.computed(function() { return searchText(); }).extend({ throttle: 250 });
searchTextThrottled.subscribe(function(newSearchText) {
// perform AJAX to MVC action to fetch search results from Google Docs
$.getJSON("/Home/Search", { searchTerm: newSearchText }).success(function(results) {
searchResults.pushAll(results);
});
});
As soon as we push results into our searchResults array, Knockout automatically renders the search results onto the page, thanks to data-binding. Sweet!
And notice how we subscribed to the throttled value of the search text. The searchText observable is updated as the user types, then the searchTextThrottled value waits for a no-typing period of 250ms before sending the query to the server. Complex UI interaction pattern, but took just 3 or 4 lines of JS code to accomplish, thanks Knockout.
Bottom line: KnockoutJS gave us the ability to write an interactive UI with ease.
Summary: It’s a great time to be a web developer!
The web is evolving and getting better. The technologies now available to us as web developers is staggering; tools like HTML5, KnockoutJS, Modernizr, RSS make it easy to build web apps that shine. And external services like IfThisThenThat, RSS, and Google Docs, makes building services that integrate with the opened web a piece of cake.
These opened web APIs grant web developers powerful capabilities not in previous generations. The opened web is coming to fruition, and we’re reaping the benefits.
Web dev for the win.