Integration Cookbook Bitext
Share!TwitterFacebookLinkedInGoogle+Email

Hey everyone, this week’s integration showcase is Bitext and this will be the last of the five sentiment analysis APIs that I’ll be writing about. The previous ones I’ve covered were Viralheat, Chatterbox, Semantria, and AlchemyAPI. For one last time, I will try to be as objective as possible and highlight all aspects of the integration from my personal experience.

Bitext

Bitext is another text analytics solution that offers a sentiment analysis API. The Bitext API doesn’t fall under the conventional definition of an ‘API’, because the exposed end-points don’t follow REST or any other API principles. Instead they utilize web forms built on an ASP.NET platform. Technically, to access their services, a user just needs to make a POST (submit) request to a remote form, which in turn responds with analysis results.

While I was integrating, I spent a lot of time fighting with the Bitext API because of problems on their side. Their documentation didn’t correspond to the reality of the service, so even copying their C# sample was not leading towards any correct results. Finally, I realized where the problem was and implemented my own API client for exposing their sentiment analysis service. The POST request looks as follows:

Dictionary<string, string> parameters = new Dictionary<string, string>();
parameters.Add("User", context.Key);
parameters.Add("Pass", context.Secret);
parameters.Add("OutFormat", context.Format.ToString());
parameters.Add("Detail", "Global");
parameters.Add("Normalized", "No");
parameters.Add("Theme", "Gen");
parameters.Add("ID", document.Key);
parameters.Add("Lang", LocaleHelper.GetTripleLanguageAbbreviation(context.Language));
parameters.Add("Text", HttpUtility.UrlEncode(document.Value.Source));

byte[] data = Encoding.UTF8.GetBytes(FormatParameters(parameters));
WebRequest request = WebRequest.Create("http://svc9.bitext.com/WS_NOps_Val/Service.aspx");
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
request.ContentLength = data.Length;

The service worked as expected, however as soon as I got a response from the server I found another problem that led me to spend hours getting a native object from the raw output returned by the Bitext service. The returned XML is formatted so the native .NET serializer is not capable to de-serialize the output as is. In order to solve the problem I implemented several regular expressions, which cleaned up the returned XML making it compatible for deserialization.

result = result.Replace("\r\n", string.Empty)
.Replace("\r", string.Empty)
.Replace("\n", string.Empty)
.Replace(">\"", ">")
.Replace("\"<", "<");

Unfortunately, as soon as I cleaned the output data and ran the test I noticed that Bitext returns analysis results using a different encoding. So, I needed to detect the encoding before deserialization in order to get the correct representation of output results. I did it using another regular expression.

Regex regex = new Regex(@"(?<=\bencoding="")[^""]*");
Match match = regex.Match(result);

Finally, I was able to de-serialize the sentiment output using the following types, which I created for mapping purposes as for any other considered services.

[XmlRootAttribute("RESULT")]
public sealed class BitextSentiment
{
[XmlElementAttribute("BLOCK")]
public List<BitextSentenceSentiment> Blocks { get; set; }
}
[XmlRootAttribute("BLOCK")]
public sealed class BitextSentenceSentiment
{
[XmlElementAttribute("ID")]
public string Id { get; set; }

[XmlElementAttribute("GLOBAL_VALUE")]
public double Value { get; set; }

[XmlElementAttribute("TEXT")]
public string Text { get; set; }
}

Overall Opinion

Generally, the service is good, however; the exposed API is a little weird. It is not an API in terms of my understanding of API. The integration with Bitext API was complex because they didn’t offer an SDK or accurate documentation. The API output was distorted and required pre-processing.

George Kozlov is a software engineering guru. He specializes in software research, architecture and maintenance. He co-founded Semantria, and is currently their CTO and go-to guy when things need improving.

Share!TwitterFacebookLinkedInGoogle+Email