choices.txt

  Overall architecture
  --------------------

  From the outset of this project the main problem I was concerned with was how
  to manage the split between Networked and Non-networked mode. I wanted to
  have very little code replication to reduce code complexity and improve
  maintainability. Therefore I conceived the following architecture. The main
  goal of this architecture is to have the majority of client and Server code
  unaware whether it is running in networked or non-networked mode. On startup
  of the application, using a factory it creates an instance of an Application
  based on the start mode (Client, Server or StandAlone). These Applications
  are responsible for performing the tasks that differentiate the modes. The
  value of this architecture is all code outside these Applications have no
  awareness of their networking mode allowing complete code re-use.

  Explanation of modes
  --------------------

  The Server Application when started presents the ServerUI which is unaware of
  its networking mode. The main problem I faced with keeping the ServerUI
  networking unaware was how to allow the server side code to be completely
  re-usable while allowing the Application to correctly publish the server
  interface. I managed this by using dependency injection to allow the ServerUI
  code call the #start() method of the Application that created the ServerUI.
  When the user clicks the Start button the ServerUI gets the instance of
  Application (in this case Server) via dependency injection and calls it's
  #start() method. The Server#start() method creates an instance of DataService
  which is then published via RMI for use by networked Client's.

  The Client Application when started presents the user with a 
  NetworkedClientUI which allows the user to enter the location of the
  networked Server. The problem I faced keeping the client side code network 
  unaware was how it would connect to the server side without knowing if it was
  using RMI or not, to solve this I let the client code get a reference to the
  server via dependency inject. When the user clicks OK the Client then gets
  the RMI based instance of DataService and publishes it via dependency
  injection for use by the ClientUI. Finally the Client starts the ClientUI
  which is unaware of it's networking state, using only the instance of
  DataService that was published via dependency injection to communicate with
  the server side code.

  The StandAlone Application, like the Server, starts the ServerUI. The only
  difference being when the ServerUI gets the published instance of Application
  it's the instance of StandAlone. In StandAlone#start() it creates an instance
  of DataService, like the Server Application. Finally, like the networked
  Client Application StandAlone then publishes the DataService instance via
  dependency injection and starts a ClientUI.

  Server side architecture
  ------------------------

  The server side architecture is composed of three layers, the DataService
  layer, the Data layer and the IO layer.

  The DataService layer is the server interface callable by clients. It is
  responsible for updating all clients when changes occur to the server data,
  managing the locking and thread safety of all reads and writes made by
  clients and finally it is responsible for any business logic that needs to be
  performed, i.e. exact match.
  The DataService layer was introduced to removed a problem I was facing. I 
  wanted clients to be able to talk to the server without having to do any
  locking. So a server side business layer was a simple solution.
  The DataService layer when constructed uses a factory method to get an
  instance of Data which it passes calls to when reads/writes are necessary.

  The Data layer is responsible for managing all the data records for the
  application. The Data object is created via a factory which retrieves the
  database location from the application Properties file and passes the
  location to the Data object. On construction the Data object uses the
  DBParser to build a cache of the data records. I decided to use a cache as I
  wanted to keep all the methods of the DBMain interface to be simple and clean
  and using a cache enabled this. The cache is used as a write through cache
  which allows for fast and efficient reads while keeping the on disk database
  constantly up-to-date. This was also a key concern I had, as I didn't want
  the database file out of sync with what was shown on the clients'.
  The Data layer also provides the locking mechanisms that allow for thread
  safety while reading/writing from the Data instance.

  The IO layer is responsible for dumb reading and writing to the database
  file. I decided to abstract the file IO to a separate layer as reading and
  writing to the database file using the proper schema can be complex. So
  having all that complexity separated from the logic of the application means
  future changes to the file schema will isolate code changes to this layer.
  This greatly improves maintainability and helps those unfamiliar with the
  code understand it.

  Client side architecture
  ------------------------

  The client side UI is split up into multiple Panels that extend JPanel, which
  are then collected into a 'Page' which is just another JPanel comprised of
  the Panels. I done this so if for instance the search functionality was to be
  replaced a new JPanel containing the new search Components could be created
  and swapped for the existing SearchPanel making extending the application
  simpler.
  Any part of the client application can make calls to the server side via the
  DataService which is published via dependency injection. This allows for
  freedom in the design of the client application as a reference to the server
  interface does not need to be passed around to all objects that make calls to
  the server side.

  Keeping Clients up-to-date
  --------------------------

  As this can be a distributed application with multiple clients running on
  multiple machines simultaneously I wanted to fix the problem of clients
  having stale data when another client updates/deletes a record. To solve this
  problem I used the observer pattern to allow clients register with the server
  for updates on startup. When the server receives a call to any of it's
  methods that change the data on the server it then goes through it's
  registered clients and updates them with the updated record.
  So far this is a simple implementation of the observer pattern however when
  trying to implement this over RMI there were other considerations to be made.
  Again like my main architecture I wanted to code once and use twice therefore
  I created a RemoteObserver interface that was capable of being called over
  RMI and it could be used in StandAlone mode without being exported to the RMI
  registry.
  With this implementation when a client (networked or not) made a change to
  the data, the server would make a call back to all clients to update them
  with the latest data. This also had the positive impact where there was no
  need to update the Client UI when editing a record. All I had to do was send
  a call the server with the update being made and it dealt with updating all
  clients. This simplified the client side code greatly as I wasn't trying to
  make server calls and update the UI at the same time.

  Data.java Locking
  -----------------

  When I was planning how I was going to implement the locking for the server
  side my biggest goal was simplicity, speed & reliable. I opted to use
  ReentrantLocks as this greatly out performs the use of the synchronized
  keyword. Using locks also allowed me more flexibility in how I manage locking
  of records. I decided to use two separate locking mechanisms, one for reads,
  updates and deletes and a separate lock for creates. The reason for this is
  it allows clients to be able to make a call to create without blocking any of
  the other read, update or delete commands giving the server higher
  throughput.
  The create lock is a single ReentrantLock, this lock is private to Data.java
  and cannot be directly manipulated by other classes. When the user calls the
  Data#create() method the thread attempts to lock the write lock. Only one
  thread can enter the create method at a time.
  The read/update/delete locking is comprised of a collection of ReentrantLocks
  one lock for each record in the database. This allows for multiple clients to
  make updates to separate records at the same time.
  However as I used a write through cache this caused issues as when two
  clients tried to write to two different records there was contention between
  the two threads for writing to the file. Therefore I implemented another lock
  in the DBWriter class that ensures only one thread can write to the database
  file at a time while still allowing threads make changes to the cache.

  Domain Objects
  --------------

  As I see it I had two options on how I was to store the data records read
  from the database. My first option was to adhere to the limitations of the
  DBMain interface and use String[]'s everywhere for storing the data records.
  My other choice was using a domain object for data storage. I arrived at a
  decision to make the best of both approaches.
  At the IO & Data layer I would keep the reading and writing of the data records
  generic, using only String arrays. This allowed these two low level tiers be
  completely generic therefore if changes to the database schema ever occur it
  should not have any impact of these two tiers once the new schema is
  reflected in the DBSchema class.
  Then the DataService layer would convert to and from a Contractor domain
  object. This gives me greater control over the data stored in the records and
  makes all the client side code more readable and maintainable as it only
  deals with these well structured Contractor objects.

  Network Communication
  ---------------------

  I opted to use RMI and my network communication device. In my opinion RMI
  offered greater code simplicity, extensibility, and allowed me to easily
  implement my desired architecture where I could reuse as much code as
  possible without having to implement the same functionality twice.
  With RMI I can have a single server interface used in both networked and
  standalone mode and the only difference in the two modes was, networked
  published the interface via RMI whereas standalone mode creates a local
  instance of the interface implementation. To achieve this with serialization
  would be extremely complicated.

  Lower Level Considerations
  --------------------------

  Create *.io package
   - Place all file IO classes into this .io package, keeping it in one place.
   - Keeps the main Data.java class clean as it can delegate work to the .io
     classes for all file reading/writing.
   - Puts all the complex reading and writing to the DB into one place.

  Read full DB into memory
   - Improves read performance drastically.
   - Makes future code changes to Data.java simpler as data look ups are done
     on a Java collection.

  Data access layer, records as String[]
   - String[] makes the DBParser.java more generic and transportable.
   - If DB scheme ever changes, no need to change how the DB is read, only UI
     and server updates.

  Server 'Service' layer uses Domain Object
   - At the service layer the server converts the String[] taken from the data
     access layer to a Domain Object.
   - Using a Domain object reduces code complexity.

  Made Data.java Package private
   - Data.java is the class exposed by the server to give clients access to the
     database info. The server should only have one instance of this class as
     all clients should go through this instance to read or write to the
     Database or else there would be no way to maintain database integrity. To
     do this a factory pattern is used to construct Data.java.

  RMI
   - RMI adds some initial 'setup' complexity, but after setup reduces code
     complexity substantially.
   - Using RMI keeps remote server calls clean, easy to understand
     (junior programmers) and easier to maintain.

  Main.java
   - Server, Client & StandAlone implement a common interface.
   - Use factory pattern to dynamically chose which to run based on command
     line flag.
   - Keeps main start class clean with single purpose.
   - Makes it very simple to have Non-networked & Networked modes.

  Client.java
   - Use of basic dependency injection to get the Data.java instance. Keeps
     Client simple & reusable as it doesn't care if it's networked or not.

  ClientUI.java
   - Single class with sole responsibility to create a new JFrame for the
     client application.
   - Makes Application extensible, as new 'pages' can be created & added
     easily.

  UI Components, JPanels
   - Extend JPanel for each Client area.
   - This keeps each UI components separate, easily maintained, easily
     replaced.
   - Sets up a good design flow to follow when adding new functionality.

  TableModel
   - Use of Swings inbuilt TableModel for displaying data.
   - The model is very simple to implement and use.

  Injection.java
   - A very basic Dependency Injection framework.
   - Allows clean and simple access to objects that need to be accessed from
     multiple classes.
   - Removes complicated passing of objects that's hard to follow.

  Server Side Architecture
   - Server side design of application is as follows,
     suncertify.db.io, Low level database file access classes.
     suncertify.db, Data access classes (required interface).
     suncertify.server, Server 'Service' layer & publicized server interface.
   - This architecture splits the server side complexity into different
     sections, allowing each layer to be focused on one task.
   - Brings all application logic to server side, making the client side dumb.
     This architecture will help when moving to a web based application as the
     web client will be a dumb client.

  Data.java Locking
   - Made use of java.util.concurrent.* package to implement locking.
   - No use of key word synchronized as it introduces a large performance hit.
   - Locking implemented using a List of ReentrantLock objects for read, update
     & delete operations & a single ReentrantLock used for locking in the
     create operation.
   - This lets multiple clients do reads, updates & deletes to different
     records at the same time. Only one Create can run at a time, but doesn't
     stop the above operations from running concurrently.

  Data#find()
   - Made search case insensitive.
   - Makes use of application simpler & faster for the end users with less of a
     learning curve.

  Data#read()
   - Does not return actual result, returns a copy.
   - This stops the Standalone application UI from modifying the cache
     indirectly without a call to the server side.

  DBSchema.java
   - Created a Class to store Database information.
   - Stores schema info, but also contains dynamic information, i.e. Total
     number of records, full byte length of a record. This makes it easy to
     read/write to database, as we have the full structure info at hand.