Re: [Gnumed-devel] Re: Gnumed-devel Digest, Vol 3, Issue 29

[Karsten Hilbert]

Ah, finally I understand what you mean.

> What is the scope of transactions? 
Currently, many EMR vendors here in Germany are working on
decreasing the granularity of transactions. The best of breed
go down to the object level using object databases like
FastObject by POET. They lock at the single object level, say,
one diagnosis entry, one billing code, etc.

The scope of a transaction for us is to be as small as we can
get to.

> For instance, is it at each page viewing change? What if there are 2
> clinicians entering data and the scope of the transaction boundary is
> patient wide, so that one clinician finishes his entry and then is
> informed his update is aborted because MVCC detects that there is a
> read-write write-read or write-write conflict.
His update isn't aborted but displayed for merging.

>> The restarted client would have to save elaborate connection
>> state information to make sure it is working against the same
>> set of databases as last time around.
> I don't understand that phrase 'same set of databases'. How does the set
> of databases in use change ? 
Well, you select another core database to connect to at startup. Or
you reconfigure your set of distributed databases and restart
your client.

> needs to be stored in a file. However, in this case, if the network
> is congested, the entry into demographics will be slowed as each
> field update must go across the network.
Agreed. But this is difficult at best to avoid.

> Another scenario is that there is a network failure or server failure,
> during the day,
In most cases you should hire more competent NOC personnel as
apparently there hasn't been any redundancy. This is exactly
another scenario where the set of databases against which I
work may change: After server A goes down and server B (the
automatic failover) is scared to death I will manually connect
to server C off-site which only receives hourly updates.

> but fortunately, the day's scheduled patient records
> have been cached on the clients.
Entirely unrealistic. The cached data will never be sufficient
if only by Murphy's Law. But, point take, some information is
better than no information (as long as it's accurate).

> Doctor's continue to see the patients,
Hm, what do I do about the odd 50 patients that come in
unscheduled (just nitpicking, I know).

> but now everyone is forced to have only exclusive read-write access at
> the patient record level. When the network is restarted, hopefully on
> the same day and in a few hours, the entered data is batch entered onto
> the server from every client, and since no one shares a patient record,
> no conflicts can occur.  A similiar scenario could occur for mobile
> computing. When a client is connected to the server, then MVCC applies.
> When a client is disconnected, it uses the local cache, and other people
> trying to access the patient record must be told someone has primary
> update rights who is not connected and a restricted  level of access is
> in force, or only read only access is allowed.
This is certainly very desirable. I suspect, though, it is
very time-consuming to implement (which doesn't mean we
shouldn't strive for it) in order for us to have a useful
GnuMed sooner rather than later. There's a difference between
doing things right and doing things hacky-kludgy vs. doing
things right on a small scale and doing things right on a
large scale. Two different domains.


> if you have MVCC( or transactions) at the table row level, (e.g
> conflicts only occur if someone is editing the same drug or past history
> item), you would probably have some sort of polling or callback to occur
> so the changes made by one doctor around the same consultation time,
> would be seen on the other doctor's screen.
Hum, ha.

gmDispatcher.py ?
gmBackendListener.py ?

> But you could also have
> checking out for exclusive access or default exclusive access if the
> application can know who is the primary updater.
SELECT FOR UPDATE ?

> Using postgresql doesn't mean you can't have local caching, you just 
> have a more flexible consistency scheme.
Sure.

>       A real life scenario is mobile computing, and network or server
> failure.    
Yes, but network/server failure isn't solved by designing
applications to survive full-force phaser attacks on Mars but
rather by providing for network/server/disk redundancy (oh, I
DO hope GnuMed will be deployed on DS9 soon ;-)


> > 1) the patient has left, the result comes back, is associated
> > with the patient's demographics which turn out to be wrong.
> > This can happen if the patient changes place of residence
> > without dutifully reporting that to his doctor. Unlikely,
> > particularly since the patient will have left a forward-order
> > with his erstwhile post office.
> The computer system can't enforce staff to check patient details are
> current. It's probably not a computer problem. 
Exactly. Hence we don't need to try to solve it that way.

>>> No clinician should assume a system records all observations if in
>>> fact it doesn't, especially in an environment where there is off-site record
>>> keeping,
>> I must admit I fail to understand the meaning in this that
>> goes beyond the obvious. I particularly don't understand the
>> connection with off-site record keeping.
>> I'm thinking of someone who relies too much on the computer records,
> and justifies not reading hand-written records because everyone is
> supposed to come back and make entries into the computer system.
Aha ! True. Upgrade to User 2.0.

> I'm trying to work out why you think local caching won't work. I'm
> assuming it violates ACID in some way; local caching might violate
> durability of updates, if no extra-ordinary exclusive locking is
> allowed.
I don't think it can't work. I just happen to think that it
may get extraordinarily complex.

> Ok , now I think the issue is , can this system have flexible
> transaction boundaries ?
> If no, transactions are row or field level, then everyone HAS to be
> connected to have a useful client and have to tolerate congested
> networks, but there is no  need for local caching for client's freezing
> and manually restarting. The client only needs to remember what state
> the ui is in ( which page and which patient it was looking at), 
> and everything is reloaded from the server like a good thin client does.
> So no cPickling needed , (not at this stage anyway)?
That's my mindset at this point along the way.

> I gather the handler collaboration is still unpopular,
> because it seems to only update non-persistent dictionary objects
> stored in lists. What about the multi -level undo feature? 
Being unpopular is less of a problem as opposed to it still
being poorly understood (at least by me).

>  I think people are wanting something like:
What level of people are you talking about ? Users or
developers ?

> the clear button inserts a new empty row into a corresponding
> table (or view).
Nothing like that. Users don't much care what happens (they
only [should] care that they can find out if need be == Open
Source). They most likely don't much enjoy "clear" buttons
either except for strict list type data. As a user *I* just want
to type into a field that's labelled "history of present
illness". There may be some data there already (coz I saw that
patient yesterday). I want to be able to add to/modify that old
data (display level modify, not db level). I want to be able to
say "that part of the old data is invalid, don't display it". I
want to be able to say "display everything history I or someone
else typed in between such and such. I want to be able to say
"this history belongs to that episode". I want to be able to
say "show me all the encounter data for that episode". I don't
friggin' care how it's implemented (from a user's point of
view). I want all the data I enter to be timestamped which may
mean several db rows per "history of present illness" field.

> Each lost focus of each text element causes
> an update of the corresponding field in the table row. 
Maybe yes, maybe no. Logical boundaries (business logic) will
not always align with the tab order. The edit area is a
perfect example for that. Each edit area 'type' represents a
well understood (?) medical activity: Writing a script,
recording an allergy/an immunization shot, doing the
'paperwork' for a referral. So, the business level transaction
boundary which starts inserts and commits data is at the time of
pressing the "do now" button of that edit area.

> All the lists are regularly polling with select statements
> to get the particular list for the patient, so any cocurrent
> entries from other users are detected. 
No. This is handled via database level notifies. The main
reason we use pyPgSQL.

> Double clicking the a row in a list view, moves the key 
> into the editarea , which then sql selects the data and
> loads the editarea.
One would want to enter the edit area on purpose and start
typing to achieve some goal. Our task is to complete medical
tasks, not maintain lists. Now, when I am in the field
"doctor/speciality to which to refer patient to" I want to
type "neur" and see all neurologists in my area (those that I
usually refer patients to at the top) AND all other doctors in
my area who happen to have a "neur" in their name somwhere.
Upon a single keyboard binding I want to be able to jump to a
list of all doctors in my area. From there to all doctors
stored in this system by yet another keybinding.

Karsten
-- 
GPG key ID E4071346 @ wwwkeys.pgp.net
E167 67FD A291 2BEA 73BD  4537 78B9 A9F9 E407 1346