Architectural Solution - Distributed Caching to improve Performance and Scalability – Part 1

A lot has been written and discussed on various forums and conferences to address SharePoint Performance and Scalability challenges. (This is quite late to write on this topic, yet I think this would be useful.) Distributed caching is one of them to achieve these two quality attributes. Until the integration of AppFabric Caching into SharePoint 2013, there was no way to achieve distributed caching in SharePoint applications. There are few third party products available to achieve this. But these products involves cost, a tougher task to convince Application Stakeholders. Moreover, if Concurrency and Cached data integrity are the primary concerns for Cached Data in an FARM environment (see Fig. 1), these third party products cannot always be an affordable solution. The article series is intended to help Architects and developers in addressing SharePoint Applications’ key quality attributes – performance, scalability and concurrency using Distributed Caching within FARM environment.

Let us have a look at the Performance, Scalability and Concurrency problems in SharePoint environments.

Problems:

Fig.1 SharePoint FARM

Performance and Scalability:

SharePoint being an intensively Database driven web application, every request involves database transaction latency. Internally SharePoint executes at least 8 SQL queries or stored procedures when user requests/adds a single item in a list. Performance and scalability problems start appearing with the increase in Data and users. Architecturally, there are very few ways in SharePoint to improve Performance and Scalability. Usually Microsoft suggests to Scale-out the infrastructure which again involves considerable cost and maintenance.

Concurrency and Data Integrity:

In general, developers in many cases resort to Data Caching in tuning the performance where they generally use in-memory caching to cache static or required data. This approach has another drawback. Considering above FARM topology, every request by the same user might not be served by the same WFE. Consequently, In-memory caching runs into the Data Concurrency issues where every WFE might not have updated cached data. Further, in an event driven scenario (List Item Add/Update/Delete), updating the Cached data on each WFE is difficult task.

Solution:

Distributed Caching is the solution to address these problems. Here goes the first approach to achieve distributed caching within SharePoint environment.

Distributed Caching using Timer Job

Fig. 2 depicts the deployment architecture diagram.

Fig. 2 Caching Service in SharePoint farm

Solution outline:

  1. Write a simple WCF service which has interfaces: Add, Get and Remove.
  2. Write a Cache (for example MyCache) class which includes all the interfaces provided by System.Web.Caching.Cache internally this class uses System.Web.HttpRuntime class.
  3. Implement the Timer Job which hosts the WCF service created in Step 1.
  4. Write a Feature which activates the Timer Job on Application Server.
  5. Make sure this Feature is not activated on the WFEs.

Sample Code:

This is just a sample code to demonstrate the approach which can be take forward to implement in the respective solutions.

Service Contract and its implementation:

Caching Object

This solution uses HttpRuntime to cache objects. Above methods could be modified add other parameters like Duration, CacheItemPriority etc. Custom logic can also be written to implement Caching mechanism instead of using .Net Framework class HttpRuntime. But this would involve writing static HashTable, a timer to remove items from HashTable based on duration… the way .Net has done internally.

Timer Job that hosts the Service

This implementation of a WCF service does not require configuration file. I have used TCP binding in this example.

Client side code on WFE

Here, “10.0.2.15” is IP address of the APP Server and 1222 is the port where service is hosted. You can write wrappers with simple interfaces around the class that will use ServiceClient to abstract details and other business logic complexities so that others can use this swiftly.

There are various Distributed Caching Topology viz. Mirror, Replicated and Partitioned Caching. If a SharePoint FARM has Application Servers’ cluster, Mirror or Replicated caching can be implemented extending this approach. Detailing here how to implement Mirror or Replicated caching is out of scope of this article series (I will probably write another blog series, but really not sure when).