Consuming URL Shortening Services

HTML clipboard

Overview

This is the first article of our series that talks about accessing URL shortening services programmatically.

Here we introduce new concepts like the REST API. We also have a brief discussion of URL shortening services APIs and how you can access them.

In addition, we are going to talk about .NET support for the REST API and tools and techniques available that would help us during our journey through the API.

A working example built using C# and WinForms is available at the end of this article.

This article is the base for all other articles. Articles other than this discuss specific services and their APIs. We will make use of code and techniques discussed here throughout the rest of articles.

Let's go!

Introduction

URL shortening services are very popular these days since web addresses tend to be very long and very difficult to be exchanged via email or other micro-blogging services like Twitter.

Today, there're tenths of URL shortening services spread out all over the web. Most of them are listed in the following articles:

In this series, we are going to talk about how to consume URL shortening services in your .NET application. In other words, we will discuss how to program URL shortening services; we mean how to access them programmatically. In addition, we will have a brief discussion of each service and its unique features.

How we will go through this discussion? This is going to be a series of articles published regularly. Each article discusses a given shortening service from the following list (updated regularly, expect new services to be added):

is.gd
twurl.nl
cli.gs
bit.ly
budurl.com
adjix.com

The preceding list may not be comprehensive, many other popular services exist. However, not all shortening services have APIs! The preceding list contains the shortening services we know that allow developers to consume their functionalities via an exposed API.

Before we start with a specific service, let's have a brief discussion of features of the API and how we can access them.

Accessing the API

Most APIs in the web are just REST (Representational State Transfer) web services. A REST web service is simply a collection of functions (HTTP endpoints) that can be used to retrieve data in a variety of formats (optional.)

Given an endpoint (function) like http://ab.c/api/shrink, we could supply the required input arguments as query strings in the URL. For example, we could shorten the URL www.google.com using a call to the HTTP endpoint http://ab.c/api/shrink?url=www.google.com supplied with the required information. It is worth mentioning that every service has its own API functions and arguments. Although they all do the same thing (shortening the URL,) they differ in function and argument names.

When you call a web function, you just end up with the results in the format used in that function (XML, Atom, JSON, etc.) The function could also return plain text! It's all about the function documentation that you should check carefully before calling the function. In addition, the returned value from a function may also contain error details if the function didn't success.

Keep in mind that API calls are limited and you can't just leave the shortening code in an end-less loop! Other limitations of specific APIs should be discussed later.

Authentication

Most URL shortening services allow users to consume the service without being registered, some of allow users to register, and others not. Many other services require users to be registered in order to use the service.

Likewise, service APIs may or may not require user authentication. Some services give the user an API key that can be used in the authentication process. Other services require user to enter his login details in the API calls. Most use the API key approach.

.NET Support

Does .NET support the REST API? Sure! As long as REST web services are just collections of HTTP endpoints, we do not need to worry about accessing them from .NET since that the BCL offers us a bunch of classes (available in System.dll) that can help with HTTP requests and responses.

In addition, we will rely heavily on classes of System.Xml.dll to handle the data returned from functions (we will make use of functions that support XML.)

If we could write something in C# that calls our fake web function http://ab.c/api/shrink, we could end up with some code like this:

C# public void string Shorten(string url)
{
string reqUri = @"http://ab.c/api/shrink?url=" + url;

WebRequest req = WebRequest.Create(reqUri);
req.Timeout = 5000;

XmlDocument doc = new XmlDocument();
doc.Load(req.GetResponse().GetResponseStream());

return HandleXml(doc);
}VB.NET Function Shorten(ByVal url As String) As String
Dim reqUri As String = "http://ab.c/api/shrink?url=" & url
Dim req As WebRequest = WebRequest.Create(reqUri)
req.Timeout = 5000
Dim doc As New XmlDocument()
doc.Load(req.GetResponse().GetResponseStream())
Return HandleXml(doc)
End Function

Encoding

If the URL to shorten have special characters like '&', '?', '#', or ' ', those characters should be handled first before the URL is sent to the shortening function.

This special handling for URLs is usually called Percent-encoding or URL Encoding. This encoding replaces unsafe characters with their hexadecimal values preceded by percentage ('%') signs.

There are many unsafe characters that should be percent-encoded including '$', '+', '&', ':', and '='. A nice discussion of URL Encoding can be found in the article URL Encoding by Brian Wilson.

Why unsafe characters are problematic? Consider the following example: we need to shorten the URL http://justlikeamagic.com/?s=twitter with our fake shortening service http://ab.c, we end up with a call to the function using the following address http://ab.c/api/shrink?url=http://justlikeamagic.com/?s=twitter. Now we have two '?'s!!!

So how can we encode URLs? Simply use the EscapeUriString() function of System.Net.Uri class it would work well for us here. If you need a full-featured encoder; a procedure that would work for all and every situation, you will need to consider creating one yourself, or you can use this:

C#
Private _chars() As String = _
{ _
"%", _
"$", _
"&", _
"+", _
",", _
"/", _
":", _
";", _
"=", _
"?", _
"@", _
" ", _
"\", _
"<", _
">", _
"#", _
"{", _
"}", _
"|", _
"\", _
"^", _
"~", _
"[", _
"]", _
"'"}

Public Function EncodeUrl(ByVal url As String) As String
For i As Integer = 0 To _chars.GetUpperBound(0) - 1
url = url.Replace( _
_chars(i).ToString(), _
String.Format("{0:X}", CInt(_chars(i))))
Next

Return url
End Function

Public Function DecodeUrl(ByVal url As String) As String
For i As Integer = 0 To _chars.GetUpperBound(0) - 1
url = url.Replace( _
String.Format("{0:X}", CInt(_chars(i))), _
_chars(i))
Next

Return url
End Function

Sample

Download links for the sample application will be available in just a few days, once we publish some other articles. :)

Where to Go Next

Now start with any shortening service you like (links are update regularly, new services will be available too):

is.gd
twurl.nl
cli.gs
bit.ly
budurl.com
adjix.com

Or else, check our URL Shortening Services tag.