ExamGuru IT/.Net Guru : Using DataSet and remoting to implement distributed cache-pattern on microsoft.public.dotnet.distributed

.NETGURU

Using DataSet and remoting to implement distributed cache-pattern

Messages

Related Types

This message was discovered on microsoft.public.dotnet.distributed_apps.
Responses highlighted in red are from those people who are likely to be able to contribute good, authoratitive information to this discussion. They include Microsoft employees, MVP's and others who IMHO contribute well to these kinds of discussions.

Post a new message to this list...

Liam Davis-Mead (VIP)

I'm interested in implementing a data access layer which uses typed DataSets
as data transfer objects. I would like for the assembler which is
responsible for managing each DTO and its corresponding DataAdapter to
maintain a static cache--a larger DataSet of the same type as the DTO. Any
record which is retrieved from the database to fill a business object should
be cached when the DTO is filled. Additionally, the assembler should attempt
to fill DTOs from the cache first (the assumption here is that the typical
business object will be a single row of some primary table in the dataset,
and if that row is present in the dataset, all related rows in related tables
will be, too). However, I want this cache to be able to scale accross
multiple application servers in a web farm, so I need to propagate updates to
the other caches.

The approach I was considering would be for the cache local to the update to
enlist the caches of the other servers in a distributed transaction along
with the database. Upon enlisting, each cache would assert a local lock
preventing updates to that particular DataSet. The database would be
updated, and if everything went smoothly, the DataSet diffs sent out to the
other caches (potentially through a reliable multicast). If none of the
caches had problems, all DataSets would approve their changes, and the
database transaction would be committed.

I have provided a much more in-depth description of the method at
http://episteme.arstechnica.com/eve/ubb.x?a=tpc&s=50009562&f=6330927813&m=476000945631.
For the sake of brevity, I only included the outline here, as I suspected
this would be a more specifically knowlegeable audience.

Has anyone heard of a similar implementation, or could provide references or
guidance for implementing a distributed cache in the data layer?

Reply to this message...

Sam Santiago

Have you looked at the Caching Application Block provided by the Pattern and
Practices Group from Microsoft:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag/html/cachingblock.asp

From reading your outline, it sounds like you are introducing a lot of
overhead and complexity in order to provide caching. It's a judgment call
whether in the long run all the effort would be truly worth it. Especially,
if you are writing your own caching layer. See if the application block
fills your needs and keep it simple. That's my 2 cents ;-).

Thanks,

Sam

--
_______________________________
Sam Santiago
Click here to reveal e-mail address
http://www.SoftiTechture.com
_______________________________
"Liam Davis-Mead" <Liam Click here to reveal e-mail address> wrote in
message news:Click here to reveal e-mail address...
> I'm interested in implementing a data access layer which uses typed
DataSets
[Original message clipped]

Reply to this message...

Liam Davis-Mead (VIP)

Sam, thanks for the reply. I did look at the CAB, and didn't find that it
was as flexible as I needed. Are you familiar with it? Perhaps I could ask
you a few questions about it, and caching strategies in general. Firstly,
the CAB contains a disclaimer that it has known consistency issues under high
loads. I'm assuming this is fixed in a newer version? Otherwise, I don't
think it will be appropriate. Secondly, is the caching agent framework such
that remoting other caches to invalidate entries possible? That's not
mentioned explicitly, although I'll see what the included documentation has
to say. Finally, the CAB is a generic object cache, correct?

The problem I have with storing datasets in a generic object cache is the
issue of multiple datasets containing references to the same table. Sure, I
can work out the expiration dependencies in the cache, but I can't reuse
portions of the datasets in the cache. In other words, if I have two
datasets that share a table, and one is cached, when I create the second one,
I want to try and fill the shared table from the cache, rather than blindly
assuming I need to hit the database. An object cache isn't going to maintain
this type of relationship or schema, but datasets will...that's what they're
designed to do.

What I want is not to cache a dataset (a representation of the data), what I
want is to cache THE DATA. I understand that repeadedly allocating datasets
could impact performance, but to me this is a separate problem, and I could
use a pool for each DTO.

It's possible I could use the CAB to cache tables. This approach might give
me the flexibility I'm looking for. Does anyone have any feedback on this
approach?

Thanks,
Liam

"Sam Santiago" wrote:

[Original message clipped]

Reply to this message...

Sam Santiago

This link might actually be a better resource to answer all your questions
since it has 6 in depth chapters:

Caching Architecture Guide for .NET Framework Applications
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnbda/html/CachingArchch6.asp

Chapters 5 & 6 might be useful to you. It is essentially an object cache.
You said you want to cache the data. If you have implemented a data access
layer then you might be able to accomplish this goal by performing the
caching at this layer. It would depend on your application architecture.
For example, if you had a central IIS server in a LAN environment that
hosted some web services and cached data on this server then all clients who
access this server would share that cached data. If your app is more
complex and you had a farm of servers hosting these web services and you
wanted to sync cache across this farm then you have a more difficult
problem. If you are caching lookup table type of data or data you that is
almost static then the problem is tolerable. My main point in my first
message was simply to make sure the effort to implement such a smart cache
would provide a benefit to your application that outweights the cost in
terms of effort and complexity - justify the ROI. Your mentioning of
introducing distributed transactions and a notification process to all
caches to refresh just raised a red flag in my mind that the benefits gained
from this would truly worth it. Getting this to work properly, debugged,
and tested has the potential to impact the project significantly.

Thanks,

Sam

--
_______________________________
Sam Santiago
Click here to reveal e-mail address
http://www.SoftiTechture.com
_______________________________
"Liam Davis-Mead" <Liam Click here to reveal e-mail address> wrote in
message news:Click here to reveal e-mail address...
[Original message clipped]

Reply to this message...

Eric Newton

Sam, I'd like to keep updated on your progress on this... your assertion is
intriguing, and maybe when I get some free time I could be of some help

In theory I would scrap typing the datasets though, as the typing usually
only helps the dev's USE the data, but if the datasets are kind of a
"transparent" caching layer then you probably dont need it but of course its
all up to you ;-)

--
Eric Newton
eric.at.ensoft-software.com
www.ensoft-software.com
C#/ASP.net Solutions developer

"Liam Davis-Mead" <Liam Click here to reveal e-mail address> wrote in
message news:Click here to reveal e-mail address...
> I'm interested in implementing a data access layer which uses typed
DataSets
[Original message clipped]

Reply to this message...

	System.Data.Common.DataAdapter
	System.Data.DataSet

ExamGuru IT Solutions - .Net Guru is owned and operated by ExamGuru, Inc., the man behind .Net Guru. If you're in the market for bespoke software or software consultancy, why not get him and his highly trained team to help? - www.examguru.net/ITCertification

Need Dot Net Interview Questions?
Ask ExamGuru, Inc. for advice and help on Passing .Net Interviews

.Net Projects
Best-of-breed application framework for .NET projects, developed by ExamGuru, Inc. and ExamGuru IT

Free .net Help
Commission ExamGuru, Inc. and his team for your next bespoke software project

FogBUGZ
The only bug tracking system carefully crafted with one goal in mind: helping teams create great software.

Awesome Tools
If you don't know about these, you're missing out... IT Certification Questions
IT Interview Questions
Free Oracle 10g Training
MCSE Boortcamp
Cisco Study Guides
Cheap Study Guides
Exact Questions
Dot Net Interview Questions
Oracle OCP
Cheap Travel
Designer Perfumes - Wholesale Prices
Free Programming Tutorials

	Copyright © ExamGuru, Inc. 2001-2006
	Contact Us - Terms of Use - Privacy Policy - www.dot-net-guru.com - www.examguru.net - www.oraclesource.net - www.itinterviews.net - www.examguru.net/ITCertification