Synchronisation Issues in vCAC Workflows and how to solve them

Warning: this post describes a feature where the developers probably might not have thought of when they designed it. Use on your own risk (as with any scripts we show you here).

When using workflows to integrate with infrastructure outside the control of vCAC (like getting the next available IP address from your IPAM system) you might run into concurrency issues – unless you only allow to run one instance of a workflow at a time. The problem here is that vCAC does not provide any means for your workflows to ‘lock’ and synchronise access to resources.

You certainly can use .NET objects like ‘Semaphores’ or ‘Mutexes’ to create locks but they only exist on a single machine (in memory). But if you have more than one DEM worker around this is not an option. Surprisingly vCO might help you here, but only if you have all vCOs in an HA evironment connecting to the same database. However -due to customer constraints- this does not always work and just for locking I would vote against setting up a vCO infrastructure.

So unless you want to set up a whole new locking infrastructure on your own you might want to fallback to vCAC as the central source and authority. And with some thinking and creativity you will find out you only have to create an object within vCAC that (or its name) must only exist once within the system. As most of vCAC entities have an id or uniqueidentifier as its primary key you have to find an object that relies on its display name, eg. ‘Announcements’.
With that it is now really easy to create a locking on your own. You just have to create a new ‘Announcement’ entity with a name of your choice and add it to the ‘Announcements’ collection. When the name already exists you will receive a DataServiceRequestException with a message about a primary key constraint violation similar to the following: “Violation of PRIMARY KEY constraint ‘PK_Announcements’. Cannot insert duplicate key in object ‘dbo.Announcements’.”
On success you have created your lock and you can do the business you have to do without any other workflow instance interfering with you. After all of your script magic is over you simply remove the previously created announcement entity.

A word of caution: Actually you could choose any entity that has a ‘string’ as its PK but you really should not use anything where vCAC would start calculations and computations (as creating a new ComputeCluster) …

One drawback with the announcements entity is that vCAC only allows for 50 character in total. So that is why I would actually create a (SHA1) hash from the requested lock name and put the real name (along with a description and session id of the caller) into the message part of the announcement. A sample code might look like this:

# $MgmtContext : DynamicOps.ManagementModel.ManagementModelEntities
# $Name : name of lock
# $Description: description of the lock and its use
$sha1 = New-Object System.Security.Cryptography.SHA1Managed;
$ab = $sha1.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($Name));
$NameHash = [System.BitConverter]::ToString($ab).Replace('-','');
$Description = "'{0}' [{1}]: {2}" -f $Name, $Host.InstanceId, $Description;

$a = New-Object DynamicOps.ManagementModel.Announcement;
$a.LastUpdate = [datetime]::Now;
$a.Name = $NameHash;
$a.Culture = ''; # part of the composite PK
$a.Message = $Description;
$r = $MgmtContext.SaveChanges();
$null = $MgmtContext.Detach($a);

On case of a failure you have to delete the announcement object otherwise subsequent calls to ‘SaveChanges()’ will fail as the context is still tracking the entity (due to the ‘UpdateObject()’ call). So you might want to put something like this into your catch{} block when encountering a

# catch
# if($_.Exception.InnerException -And
#   ($_.Exception.InnerException -is
#   [System.Data.Services.Client.DataServiceRequestException]))
    $r = $MgmtContext.SaveChanges();

Waiting on such locks is a different story. In the end it actually comes down to a loop with a balanced wait/retry mechanism as there is no other way to get notified.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: