IT's Dilemma

After a very long hiatus from blogging, I’m back online.  Going forward, I would expect this blog to have a very similar tone to its previous incarnation.  That is, technically focused with a business bent.  I have studied and have a passion for both technology and business, but above all, I desire to bridge the divide between the two camps.

I blog for several reasons. First and foremost, I use blogs to solve many of my day-to-day issues.  This is a way for me to give back to the community and perhaps help solve problems for another user of blogs.  Secondly, I find some validation in blogging.  When I am working in a particularly obscure problem space, or one that doesn’t have any local validation, there is always someone on the Internet ready and willing to validate (assuming the post was valid).  Finally, blogging connects me to other like-minded (or at least similarly-tasked) professionals, which is always good for fleshing out thought patterns.

Status Update

A number of people have asked me what ever came of the software we were working on last year.  In a nutshell, we had a significant decision to make: move incrementally from Cisco Unified Contact Center Express (UCCX) to Unified Contact Center Enterprise (UCCE) or take a huge leap to Microsoft Office Communications Server (OCS).  We eventually made the (heart-stopping) decision to move to OCS, for a number of reasons:

1. The primary motivator in our decision to work with OCS was the fact that we have a number of talented .NET developers on staff.  Doing custom development work against other contact center systems frequently requires specialized training; doing custom development work in the OCS space was something we were able to pick up without any training.
2. The capital expenditure for UCCE was in the millions, the capex for OCS was 10% of that.  The scale expense per employee for UCCE was thousands of dollars, the scale expense per employee for OCS was 20% of that.
3. Maintaining UCCE requires specialized hardware and staffing needs.  Our OCS deployment uses Dell blade servers that we’re free to upgrade without consequence.  We are able to use our existing network operations staff to maintain the OCS environment.
4. UCCE made it difficult at the time to hang users off the Internet without specialized connections (VPN and/or QoS).  OCS did not require a VPN, and Microsoft recommends that OCS implementers not implement QoS unless it becomes necessary.
5. Microsoft unquestionably has the commitment to invest huge amounts of resources going forward.  We knew there would be some pain points with OCS 2007, and there were.  However, the release of OCS 2007 R2 has already improved some of those pain points, and has added some very interesting new features.

The end result is that we were enormously successful.  To be sure, there were a number of excruciatingly painful things we found – but most of our real pain points were with Speech Server 2007, which we used as an IVR.  Specifically, we found that our IVR would frequently throw exceptions in internal code, which would cause unpredictable results in the workflow.  We have patched our way through most of these issues, but we consider them to be workarounds, not solutions.  We have high hopes for UCMA 2.0 improving significantly in these areas.

The other tremendously painful things we found were that audio conferences do not support DTMF or transfers.  Our call workflow would add a recorder endpoint to a conversation as it was transferred to an internal employee, making the conversation a conference.  This made it impossible for us to bring in an external party that required DTMF input (for instance, another IVR) or fully transfer that call to an external party.  Rather, the employee would add the external party and drop off the line with the end effect of the recorder remaining on the line for the duration of the external parties discussion.  Early in the season, we didn’t realize (nor is there good documentation) on the fact that if there are no internal parties still involved in the conference, the call will be terminated after x minutes.  This makes sense as a security feature, but we wound up dropping a number of calls that we had transferred to one of our partners.  It was messy.

Still, I would reiterate that we were enormously successful.  Those areas of OCS that weren’t pain points worked phenomenally.  We got into a scalable, manageable enterprise voice solution for a small capex.  And most importantly, we wrote Complemax.

Complemax

When I was blogging last year, I expressed frustration with a number of existing contact center matching algorithms.  There are three characteristics of traditional contact center matching algorithms that I really struggled with: first, most of the algorithms out there have a queue-based approach.  I hated that, because there are simply some calls that are more important than others.  The queue-based solution to this is to create a unique queue that is shorter than other queues, but you still don’t achieve infinite flexibility – rather, you have a queue permutation problem – how many queues are you willing to support?  Secondly, most of the algorithms use a skill-based mechanism.  For the most part, I support skill-based mechanisms, but many implementations of skill-based routing have the same limitations in scoring – scoring happens in buckets rather than on a fine scale.  Finally, and most importantly, all of the algorithms I looked at (as far as I could tell) considered scoring only from the point of the call.  It’s hard to explain this in words, so let me try to present a couple of pictures.

[Disclaimer: Please take the following with a grain of salt and do your own research.  I’m not a contact center expert, and I detest people who misrepresent something without qualification.  I don’t believe I’m misrepresenting the predominant algorithms in use in contact centers today, but I could be wrong.]

Skills-based routing (fully queued)

This is, in my opinion, almost the worst possible form of routing.  The only worse form of routing would be straight queue-based routing that doesn’t even take skills into account.  In the case of skills-based routing, agents are dumped into skills queues as they become available.  If an agent has multiple skills, they can theoretically be put into multiple queues and an additional algorithm will run behind the scenes to determine which call an agent should get if (s)he is first in line in multiple queues with calls waiting.

Calls have a corresponding skill queue.  When a call is waiting in its skill queue and an agent is available in the corresponding skill queue, both are popped out of the queue and connected.  The deficiencies of not utilizing game theory are even more apparent here.  If agent 2 in the above picture is much more capable of skill 1 than agent 1, nothing in the algorithm will allow agent 2 to take the call instead.

Skills-based percentage routing (partially queued)

In this form of routing calls, an agent registers with a skills “queue” as with straight skills-based routing.  Unlike the former scenario, however, when a call is waiting, the algorithm will match the call to the agent with the highest score or percentage in that skill.  This means that the agent half of the skill queue is not really a queue as it is a grouping of available agents.

This form of routing calls clearly has some advantages over skills-based routing.  First and foremost, the caller is likely to get someone more experienced in their skill.  The agent is more likely to get a call (s)he can deal with.  Other agents, who may be stronger in other skills, may be more likely to get calls they can deal with.  Agents are encouraged to improve their skills – if they don’t, they don’t get calls.

However, this algorithm still has some major deficiencies.  For one thing, it’s really hard to represent a multiplicity of skills needed to handle a call.  In our environment, we have scenarios where we consider many factors – whether there is a language match, a licensing match, whether the caller has spoken to the agent before, what role the caller needs to talk to, etc.  In the worst case, a skills-based percentage routing would have to have the full permutation of skills queues to represent all of our skills combinations.  That means if we support 2 languages and 50 state licensure, we’re already up to 100 queues.  More importantly, this algorithm still doesn’t attempt to maximize the efficacy of the contact center as a whole.  Just because an agent is the best match for a call does not mean the call is the best match for the agent.  If that doesn’t make sense, make sure you keep reading.

Complemax routing

Because of my frustrations with the existing algorithms in contact center routing solutions, and because we were working on deploying OCS (which doesn’t have an automatic call distributor or ACD), we seized the opportunity to fix what we perceived as a problem.  I believe that most, if not all, routing algorithms in current usage take the wrong approach.  Before you write that off as arrogant, let me explain.

The first point to understand is that we want to use the term matching as opposed to routing.  Complemax looks for tight matches between agents and calls.  That matching relationship can be built of any number of factors.  We currently have about twenty different factors running in our environment, and we pretty consistently get solid matches.  Where we don’t, we try to understand which factor is adversely affecting the matches or what needs to be added or removed to make better matches.  Once we create those matches, we build a representation of a grid and start to analyze that grid for “best matches.”

This is where things get really complex.  I’m going to skip a lot of the details here not only because this intellectual property belongs to my company, but because I’d lose the interest of nearly everyone who has actually read this far.  Suffice it to say that the problem is considered NP-complete – that is, there is no known algorithm that can consistently solve the grid in polynomial time.  To compensate for the NP-complete barrier, we used game theory and Pareto frontiers to narrow the set of possible solutions.

The end result of Complemax is best understood by looking at the diagram above.  In that diagram, we can clearly see that the best match for caller 1 is agent 2 and that the best call for agent 2 is caller 1.  So why shouldn’t we match agent 2 to caller 1?  Any human can tell you, if they look at the chart for a bit, that the most efficacious course for the contact center is not to match agent 2 to caller 1, but to match agent 2 to caller 2.  This is because agent 4, while not quite as good of a match for caller 1, is quite capable of handling the call – nearly as capable as agent 1.  Simply stated, we look at a full mesh of agents to calls, their “match scores”, and we try to maximize the sum of the match scores.

I will give details in a future post about how well this solution worked for us, but the preview is that we handled almost 20x the calls we did the previous year, and we’re still in business.  It worked.

Going Forward

I have been working a lot recently on catching up on some sorely missed professional development.  I consider blogging to be part of that professional development, and I do plan to start blogging regularly again.  I hope this will be a venue where new ideas and new theories can be thought out, exposed, and hopefully receive some criticism eventually.  We have many plans now past Complemax, and I’ll start to talk about some of what we’re doing now as I get time to write.

This entry was posted on Friday, March 6th, 2009 at 5:52 am and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.