The sipsorcery web site is now running from the Windows Azure cloud. Previously the web site was running from one of the EC2 SIP server instances which wasn’t ideal as if that SIP server had an issue it meant it was not possible to access the web site.

There are no configuration changes required to use the Silverlight client on the web site, it will still be accessible on the same URL’s sipsorcery.com or www.sipsorcery.com. However if anyone had the previous sipsorcery.com redirect to https://www.sipsorcery.com/sipsorcery.html bookmarked they will need to modify it to http://www.sipsorcery.com/. If anyone is wondering why the page is no longer available over https it’s because it makes it neater for Windows Azure to have two applications, one that hosts the web services on an https end point and one that deals up static pages on an http end pont. In the new deployment even though the Silverlight client is accessed from an http endpoint it will still use the web services over https so is secure as far as web service communications from the client to the server.

A change is required for anyone using the call management service which was previously available on http and https end points and is now only available on an https endpoint. The URL for web callbacks is now https://www.sipsorcery.com/callmanager.svc/webcallback?user=username&number=1234567.

It’s always been on my todo list to spend more time with Ruby with a big motivation being able to write some more powerful sipsorcery dialplans. Last week I got an incentive to finally do it when I came across a Ruby on Rails cloud platform called Heroku. Heroku operates on top of Amazon’s EC2 but rather than dealing with EC2 instances and images they abstract that all away and allow programmers to work at an application layer. The Heroku experience is close in some respects to Microsoft’s Windows Azure cloud platform, which I’ve also been spending a fair bit of time with. There’s pros and cons to each of course, Heroku’s is a lot more polished, a lot quicker to pickup and a lot quicker to deploy with while Windows Azure is more powerful given that the .Net framework is more comprehensive and has deeper hooks into the underlying Windows OS than the Ruby on Rails software does. In fact it would probably be a better comparison to compare an ASP.Net MVC web application hosted on Windows Azure to Heroku’s offering. The same pros and cons listed previously still apply and for a programmer unfamiliar with either Heroku and Ruby on Rails have a much smaller learning curve but in my opinion an ASP.Net MVC application is more powerful and manageable.

What’s all this got to do with sipsorcery?

As part of my exploration of Heroku & RoR I thought it would be worthwhile to get a basic web app deployed that would hook into the sipsorcery web services. It would give my exercise of attempting to learn RoR a goal and maybe it could be the starting point for another programmer out there to put together a non-Silverlight interface for sipsorcery, something which comes up regularly on the sipsorcery forums.

The sipsorcery web services have always been available via a SOAP API although as far as I know nothing except my Silverlight client has ever consumed it. SOAP APIs are out of fashion these days though and REST and JSON are now the flavour of the month, they are also easier to use so that’s good news. I have exposed a few sipsorcery methods over REST, specifically the login and get SIP account methods, and it was there that I wanted to hook up from a Heroku RoR app.

The result of my efforts is at https://sipsorcery.heroku.com/auth/login. It’s a completely bare bones application, absolutely no effort has gone into the user interface, that lists the ID’s, usernames and domains of the first 10 SIP accounts belonging to a sipsorcery user. I’ve only been using RoR for a week so I’m not fully confident about the application being fully secure but the two communications channels, browser to the Heroku app and the Heroku app to the sipsorcery REST API are both SSL, so I’m satisfied enough to use my own sipsorcery username and password with it.

If there are any RoR programmers out there that use sipsorcery and are interested in a non-Silverlight interface my very crude code for the above application is hosted on github.

The main concept of the RoR app is the connection to the sipsorcery REST API. The Ruby code to accomplish that is very succinct and I’ve posted it below for anyone interested. The user and pass values need to be replaced with valid sipsorcery account details and if the code works it will dump SIP account details as a block of JSON encoded data which looks very messy but is easy for a programmer to work with.

require 'httpclient'
require 'rexml/document'

puts "starting json test..."

baseURL = "https://www.sipsorcery.com/provisioning.svc/rest/"

client = HTTPClient.new
resp = client.get_content("#{baseURL}customer/login?username=user&password=pass")
authID = REXML::Document.new(resp).elements[1].text
puts "authID=#{authID}"

sipClient = HTTPClient.new
resp = sipClient.get_content("#{baseURL}sipaccounts?count=10", nil, "authID" => authID)
puts "SIP Accounts=#{resp}"

I’ve just uploaded an alpha release candidate of a new local version of sipsorcery (local indicates it’s designed to run on a PC and is suitable for a small number of users).

The new version can be downloaded from codeplex.

It’s best to post any questions or issues with the release candidate on the dedicated forum.

I plan to put together some release notes, update the documentation and incorporate any feedback that may come up and will then make another release.

There have been 5 outages of sip1.sipsorcery.com in the last 24 hours which is verging on the ridiculous. I’ve even swapped IP addresses between sip1 and sip2 to see if the issue was constrained to one physical instance but that didn’t work.

I’m not getting much love from Amazon support Access outages seem to be caused by traffic volumes so at this point I’m not hopeful for smooth operation of sipsorcery.

My hypothesis is that an Amazon network element is blacklisting the sipsorcery server due to its traffic profile of lots of small UDP packets or something. The evidence being that only the 174.129.236.7 server ever fails and it’s the one with 80% of the traffic. Also from the server logs I can see that the server doesn’t crash and the sipsorcery servers are keep running trying to talk to the outside World it’s just that no network traffic can get in and out.

The worry is that once the primary server fails the traffic levels will build up on the secondary server and it will get blacklisted as well.

Along with playing around with Tropo to test out VoIP applications where sipsorcery’s signalling only design is insufficient I have also occasionally fired up an Asterisk EC2 instance (thanks to Voxilla for bundling up an Asterisk EC2 AMI to make things easy). Each time I would customise the Asterisk dialplan to test out whatever it was and then take the instance down a few hours later. The great thing about sipsorcery is that it saves me having to configure SIP accounts or providers on the Asterisk system. I’d always thought it would be great if someone else did all the grunt work for me and offered Pay As You Go hosted Asterisk instances and saved me even the small amount of time I spent bringing my own instance up. There are lots of hosting companies out there that will host Asterisk instances but the key for me is the Pay As You Go piece, I might only want to test something out once every three months and I’m not interested in paying a monthly charge.

That’s why I’ve been somewhat interested in Cloudvox’s offering. It’s the first time I’ve seen hosted cheap hosted Asterisk instances. It’s great that they provide free “Hacker” accounts so you can at least make sure it’s not a snake oil product. Above that the cheapest plan is $6/month and calls at $0.06 per minute. Even though I’m not a fan of the monthly charge $6 is pretty cheap and it looks like you can switch back to the Hacker account easily if you know your account is not going to be in use for a while. The $0.06 per minute is not so appealing and it’s still a little bit fuzzy to me whether SIP-to-SIP calls are billable or not, apparently they are but I found that it depends on how they are dialled as to whether they show up on my account as billable or not (note I’m not trying to find ways around the cloudvox billing rather I’m trying to work out if SIP-to-SIP calls are free).

Cloudvox have two APIs for building applications although for some reason they have included SIP+RTP in the HTTP vs AGI comparison which doesn’t really make any sense and must be awfully confusing to developer new to VoIP. The HTTP API is very basic but is the easiest to get up and running with. I found it counterintuitive to begin with but that’s because I was so used to the concept of dialplans, VoiceXML and Asterisk that I was thinking of it as a way of simply loading the Asterisk dialplan whereas the approach is for the Cloudvox Asterisk servers to receive a call, send a HTTP request for a small number of Asterisk commands to a web server somewhere, execute those commands and return the result via another HTTP request, it makes perfect sense to me now. An example of an ideal task for the HTTP API is one the same I mentioned in the Tropo Transfer post about a user dialling into sipsorcery, somehow entering a number and then dialling that number out through one of their providers. The HTTP API would be perfect for getting Asterisk to play a prompt, collect some digits and then return them to sipsorcery via a HTTP request.

The second API revolves around the Asterisk Gateway Interface. Again I got somewhat confused this time by the fact that the emphasis is put on some pre-canned AGI servers like Adhearsion or the Asterisk::AGI Perl module. My confusion arose from thinking I needed to somehow connect to an AGI server Cloudvox had somewhere or that there was some special configuration on Cloudvox’s end that only allowed it to work with specific AGI servers. In the end the situation was again a lot simpler and Cloudvox don’t care what the AGI server is and all the examples and configuration guides are targetted at the common ones on the assumption, most likely correct as well, that most people will use one of them. Once that light bulb went off for me I dusted off my own C# AGI server, written many years ago, and had calls coming into it from the Cloudvox Asterisk servers in no time at all.

I’ve checked the very rudimentary C# AGI code into the sipsorcery code base, I had to strip out all the business logic so it’s really bare bones, it’s called SIPSorcery.Asterisk.FastAGI and is in the sipsorcery-servers directory. I don’t know when the next time I’ll get fired up to do anything else with it but it’s there as a reference should any other C# developers be interested. Generally I get all excited about integrating with new VoIP platforms but once i can play the monkey sounds or equivalent I consider the integration proven until something else crops up.

Tropo is a hosted voice application platform that I and a number of other sipsorcery users have messed around with at various stages. At this point I would say Tropo is definitely the pick of the bunch from the voice application platforms I have tested out and some of the stuff that can be done in a couple of lines of script, such as speech recognition or text-to-speech, is pretty amazing.

One glaring ommission for me and again other sipsorcery users has been the lack of an elegant way to transfer calls away off the Tropo servers. Here’s a few posts from the Tropo Forums about transfers:

  • Call and Transfer (from jack9901)
  • transfer API
  • And I posted the same query regarding blind transfers at some point as well but it must have been cleaned out

The usage pattern a lot of people are after is to dial into the Tropo platform, do some processing, such as ask the user to enter a number, and then transfer the call back to a different system to do some different processing, such as forward the call to a 3rd party SIP provider. Tropo does support attended transfers but that has two big shortcomings:

  • 1. Because Tropo will ultimately be charging by the minute having their servers bridge the whole call after any application processing is done is an unneccessary expense,
  • 2. Adding an extra media server into the call path always introduces call quality concerns. Even if Tropo’s systems and network connections are the best in the World it’s extra latency on the call that’s not needed.

After the recent work I did to support transfers on the sipsorcery servers I started thinking about the Tropo problem again and realised there was a nice elegant solution. Instead of initiating the SIP transfer from Tropo I could get Tropo to send a HTTP request to sipsorcery and have it do the transfer instead. So that’s what I’ve spent the last few days doing and the good news is it works perfectly (at least for me).

The steps are:

  • Create a new dialplan on your sipsorcery account called transfer, this is the dialplan the call will end up in when it comes back from Tropo,
  • In your sipsorcery diaplan set the From header user to something that can be used to uniquely identify the call:
        when /^232$/ then sys.Dial("999142xxx@sip.tropo.com[fu=uniqueidxxx]")
    

    The From header user can be accessed in the Tropo system with $currentCall.callerID,

  • In your Tropo application the blind transfer needs to be initiated using a HTTP request a Ruby example I’ve used is:
    require 'net/http'
    
    answer()
    say "hello world"
    transferResult = Net::HTTP.get_response(URI.parse("http://sipsorcery.com/callmanager.svc/blindtransfer?user=username&callid=#{$currentCall.callerID}&destination=1234")).body
    log transferResult
    say "http://202.6.74.107:8060/triplej.mp3" # Play some audio while transfer is being carried out.
    

    The new blindtransfer web service method takes three parameters:

    • user must be your sipsorcery username,
    • callid is used to identify the call leg that’s being transferred out of the call. Tropo doesn’t seem to provide much access to the incoming call headers so I’ve tailored the sipsorcery end to be able to look up the call leg based on the From header URI user. That’s why it needs to be set to something reasonably unique in your sipsorcery dialplan,
    • destination this is the value that will end up in the sipsorcery transfer dialplan as req.URI.User.

For once the whole thing is easier than it sounds and I think for those people already familiar with sipsorcery and Tropo it will be an easy job to use the new mechanism. The great thing about it is that there is now an elegant way to solve the challenge of allowing a user to dial in and enter or say their number and then dial that number out through a different provider.

Upate: There’s a bit of confusion around what needs to happen to take advantage of this latest fix.

  • 1. NO changes need to be made to SIP accounts or the SIP account a particular ATA/Phone is using,
  • 2. A change does need to be made to the registration contact being used for a provider IF it’s used for Google Voice callbacks AND the contact username is NOT the same as the web login username.

As people quickly discovered the new redundant sipsorcery deployment did not play happily with matching callbacks on Google Voice calls. The reason is that Google Voice calls from sipsorcery are not a SIP call at all but instead are initated with a HTTP request to Google Voice which generates a callback to a 3rd party SIP provider such as Gizmo (via SIP) or SIPGate (via PSTN) which if the provider is configured correctly forwards the call back to sipsorcery. If that all goes smoothly the sipsorcery server will receive the call and attempt to match it up to the SIP call that initiated the whole thing. Messy, yes; complicated, yes; one big hack, yes; since Google Voice don’t currently provide a public SIP gateway this mechanism is the only way to use Google Voice with sipsorcery, not that that’s a major goal of the project but it seemed like an interesting challenge at the time.

Anyway the introduction of a dual server deployment at sipsorcery meant that the forwarded call from the 3rd party SIP provider could now arrive at either server. That’s not an issue for pure SIP calls and the sipsorcery application servers are smart enough to know which proxy to use when forwarding to registered SIP accounts. However it is an issue when waiting for the Google Voice callback since the originating SIP call exists on a single specific server and if the forwarded callback does not arrive at the same server they won’t be matched up and more than likely the forwarded call will get rejected or cause a different phone to ring.

I have now put in a workaround (hack to a hack to a hack sort of thing) that will allow the callbacks to be correctly matched up. The caveat is that the forwarded call from the 3rd party SIP provider MUST be to the username of the owning account that is it can’t be to any old sipsorcery SIP account it MUST be a SIP account that has the same name as the username that’s used to login to the web site.

The new mechanism relies on telling the sipsorcery SIP proxies, that sit in front of the application servers, that the next call for a specific username should be directed to a specific application server. The reason the SIP account must be the exact same as the owning username is that the SIP proxies don’t have time to go looking things up in the database, they are designed to get the SIP packets to and from the other SIP servers as quickly as possible and introducing database checks would slow them down dramatically.

So the mechanism isn’t perfect but then again the original sipsorcery Google Voice call approach is not a beacon of purity either. Hopefully it should be enough to get people by.

With the SQL Azure migration complete and the second sipsorcery application server similarly bedded down now is a good time to provide the configuration options for SIP clients to avail of the new redundancy measures.

The quick steps to take to configure your SIP client are:

1. Do nothing.

In the majority of cases there will be nothing at all needed on the client side to allow it to failover between the two sipsorcery SIP servers. The sipsorcery SRV records are now configured for both sipsorcery.com and sip.sipsorcery.com (using either is exactly the same, I typically use just sipsorcery.com because it’s less typing).

For the failover mechanism to work a SIP client MUST support SRV records (RFC 3263: Session Initiation Protocol (SIP): Locating SIP Servers to be official about it). It’s also necessary that one of the two hostnames mentioned above, I’ll repeat them because they are critically important sipsorcery.com or sip.sipsorcery.com, are used and NOT an IP address or the hostname of one of the individual SIP servers.

There are still a few minor issues I am working on sorting out such as matching up GoogleVoice callbacks, which can now arrive at either of the two SIP servers, and fixing the event logging for the Silverlight console so that it gets the events from both server rather than just a single one. Apart from those issues the new sipsorcery deployment is working very well and given that it took 5 or so months of development work to make the changes to allow a multi-server deployment not to mention the work on Amazon’s SimpleDB that ended up not being used I am so far pretty happy with the way things are running.

The single point of failure in the sipsorcery system is now the Microsoft SQL Azure database platform. SQL Azure has a monthly SLA of SLA of 99.9% which would translate to 43.2 minutes of downtime in a 30 day month. That’s actually a pretty low level of reliability for a telecoms type service where five nines (26 seconds per month) is typically the bare minimum and six nines (2.6 seconds per month) is desired but hopefully the SQL Azure service will exceed expectations. Incidentally there has been an outage already in January, on the 17th between 0503 UTC and 0519 UTC the sipsorcery applications were able to read but not write to the SQL Azure database so that’s 16 minutes of the 44.6 minutes (January has 31 days so the SLA allows an extra 1.4 minutes of downtime) used up.

The data migration to SQL Azure has been completed and the sipsorcery.com web site is now available. The migration went very smoothly but if anyone notices any issues please post on the Forums. The MySQL database has been switched off, although it will be left in situ for the time being, and the sipsorcery.com service is now running between the Amazon EC2 and Microsoft Azure clouds.

I will be migrating the sipsorcery.com database from its current MySQL instance to an SQL Azure database between 0000 and 0400 UTC on 13 Jan 2010. During that time the sipsorcery.com web service will be unavailable and updates to SIP accounts, SIP providers and dialplans will not be possible.

I will post a a notification on this blog once the migration is complete.