Readme.txt 
Pointrel Data Repository System
October 8,2001
Paul Fernhout
pdfernhout@kurtz-fernhout.com
http://www.kurtz-fernhout.com

* What is the Pointrel Data Repository System?

The Pointrel Data Repository System is a variant of an Entity-Relationship model database. The Pointrel system provides a way to easily handle loosely structured data stored on disk, like for INI files, version control systems, bug tracking systems, or simple AI type applications. It takes an approach to data storage which emphasizes flexibility over speed. It also emphasizes storing new information for the long term over modifying or deleting old information. It hopefully makes it easier to build new layers of abstraction and indexing over old data. The Pointrel Data Repository System bears some resemblance to the ROSE/STAR system described by William Kent in his book "Data & Reality".

In a nutshell, the Pointrel Data Repository System helps you build associations which define relationships between entitites. These associations are essentially triadal links between things indicating one thing is linked to a second thing in a way defined by a third thing. The simplest way to use such links is to make the equivalent of object properties or a dictionary, such as "Fluffy weight 20kg" which if a dictionary would be Fluffy["weight] = "20kg". However, Pointrel differs from a dictionary in that is supports queries like one for all dictionaries which define a weight of 20kg or all relationships between "Fluffy" and "20kg". Triads are all defined within a specific context that gives meaning to the associations (making triads actually have four fields). The context allows triads to be handled within an archive in a somewhat more modular fashion using them as filters, since you can easily ignore triads not in the context of interest. All fields of a triad are indefinite length binary strings -- so they could be anything from "foo" to the contents of a binary file. 

* What is a simple example of the Pointrel API being used?

See "fluffyExample.py" for an example of using the simplified global function interface.

Here is an excerpt from that file:
  from pointrel20010915SimpleInterface import *

  Pointrel_initialize("myArchive")

  Pointrel_startTransaction()
  Pointrel_add("simpleexamplecontext", "Fluffy", "weight", "20kg")
  Pointrel_add("simpleexamplecontext", "Fluffy", "color", "beige")
  Pointrel_add("simpleexamplecontext", "Fluffy", "teeth", "pointy")
  Pointrel_add("simpleexamplecontext", "Fluffy", "teeth", "nasty")
  Pointrel_add("simpleexamplecontext", "Fluffy", "preferred food", "Knights who say 'Nie!'")
  Pointrel_finishTransaction()

  string = Pointrel_lastMatch("simpleexamplecontext", "*", "weight", "20kg")
  print string
  # string would be --> "Fluffy"

  string = Pointrel_lastMatch("simpleexamplecontext", "Fluffy", "weight", "*")
  print string
  # string would be  --> "20kg"

  string = Pointrel_lastMatch("simpleexamplecontext", "Fluffy", "teeth", "*")
  print string
  #string would be --> "nasty"

  list = Pointrel_allMatches("simpleexamplecontext", "Fluffy", "teeth", "*")
  print list
  #list would be  --> ["pointy", "nasty"]

* What is a more complex example of using the Pointrel system?

See the file "pointrelMemex.py". It implements a version of the Memex archiveing system proposed by Vannevar Bush it the 1940s.

Here is a typical API useage for the more complex OO API:

  repository = PointrelDataRepositorySystem(archiveName)
  repository.startTransaction()
  repository.add(context, a, b, c)
  repository.finishTransaction()
  string = repository.lastMatch(context, "*", b, c)
  string = repository.lastMatch(context, a, b, "*")
  list = repository.allMatches(context, a, b, "*")
  string = repository.generateUniqueID()

* What is the license? 

BSDish. See license.txt for details.

* What versions of Python is it for?

This code was developed primarily under Python 2.1 under Windows 2000. 
It has also been spot tested under WinNT 4.0 under Python 1.5.1. with changes made to run there, however, the primary focus is Python 2.1.

* How reliable is the Pointrel System?

The Pointrel system should not be relied on for mission critical systems yet. It has not undergone enough testing, expecially in terms of multiple simultaneous users of the same archive. It may be adequate at this point for INI file handing or similar non-critical applications if you do some testing on your own for suitability in your particular circumstance -- however I cannot guarantee that. The software comes with NO WARRANTY.
I do not yet use Pointrel for anything mission critical. One reason for this release is in hopes people will try it, provide feedback, and perhaps make suggestions for improving its reliability. I will say that ensuring the long term integrity of the data stored is a high priority.

* What is included in this release?

readme.txt -- this file.

Pointrel20010915.py -- The core code of the Pointrel data repository system.
pointrel20010915SimpleInterface.py -- A simpler interface to the core code.

fluffyExample.py -- A simple example.

pointrelViewerTK.py -- TKInter version of a viewer to look at an archive.
pointrelViewerTKSimple.py -- Same as pointrelViewerTK.py but using simpler Pointrel calls.
PointrelViewerWX.py -- Same as pointrelViewerTK.py but in wxWindows.

pointrelToDoApplication.py -- Stores "to do lists" using versioning.

pointrelMemex.py -- Implementation of Vannevar Bush's Memex proposal (from 1940s).
memex.archive -- Demo archive with the Pointrel source files loaded into it and a few trails.
(The log file was deleted to save space and show you don;t need it to use an archive.)

pointrelNotebook.py -- A simple versioning journal (can also run entered Python code).
notebook.archive -- A sample notebook archive.
notebook.log -- the log file.

pointrelSimpleFun.py -- Another simple exmaple using the simpler interface.

pointrelDrawerWX.py -- Simple program to save and load drawings of circles.

pointrelFromNetscapeEmail.py -- Imports a netscape mail file into a Pointrel archive.
pointrelFromNetscapeEmailSimpler.py -- Similar to pointrelFromNetscapeEmail.py
pointrelEmailViewer.py -- Displays a mail archive imported by previous files.

pointrelIntegrityChecker.py -- Checks consistency of an archive.

pointrelRecover.py -- Puts archive in consistent state (not really needed).

uniqueID.txt -- file used by repository for collision avoidance (which is remade if deleted).

* Why is there a version number in the Pointrel "pointrel20010915.py" file name?

The Pointrel Data Repository System is intended for the long term storage for data. It is likely better systems for putting the data on disk will be created from tiem to time with potentially incompatible formats. By including a version number in the library name, it will be possible to import both versions fo a library to migrate data to new formats as desired. The version number is also included in the archives themselves. The version number reflects the date on which a non-backwardly compatible change was made to the file format. It does not reflect the last date for change to that version of the system in terms of API improvements or bug fixes.

* What are the differences from previous Pointrel versions?

This version focuses on ease of use. It includes more examples. It changes the way triads are defined to use only string items instead of internal hidden pointers. It adds the notion of context to a triad. It also allows multiple archives to be searched at once as a repository.

* How do you delete things in Pointrel?

The Pointrel API does not have a delete function -- or even a function to change data. Everything added to an archive is there indefinitely. However, one can build higher levels of abstraction that allow deletions and changes to application data. The example "pointrelToDoApplication.py" shows how items can be deleted in practice -- essentially by setting a deleted flag associated with the item and then removing flagged items from a search result before using or displaying the list of items. For permanent deletion of materials, an archive needs to be cloned by copying the parts you want and leaving deleted material behind. The example "pointrelMemex.py" includes code to selectively export and import part of an archive. 

* What are the other files besides an archive file that have the same name as the archive but different extensions?

An ".archive" file stores all your data. A ".log" file records each triad as it is added to the archive. The log file can be deleted or supressed, but it is potentially useful as a data recovery if the archive were to become corrupt. The ".recovery" file is used for error recovery for abandoned transactions. The ".lock" file is used for supporting multiple users sharing access. The ".recovery" and ".lock" files are normally created and deleted in the course of operations on the archive. If code starts a transaction and then fails for some reason (like an exception) the ".recovery" and ".lock" files will get left behind. When the archive is reopened, if a ".recovery" file exists and the file is not otherwise in use (indicated by a stale ".lock" file more than ten seconds old since the last modification time), the archive is rolled back using the ".recovery" file to a state prior to the start of the last failed transaction.

* What are the API functions "PointrelDataRepositorySystem.addToActive(archiveName)" and "PointrelDataRepositorySystem.removeFromActive(archiveName)" for?

While there is no direct example of it, a repository may have multiple archives simultaneously open. All are examined during searches. However, only one archive will be added to at a time -- which is set using PointrelDataRepositorySystem.setArchiveForAdding(archiveName). Note that you could arrange to have the archive added to not be one of the ones searched. By default, a repository uses only one archive having the name passed in when the repository is created.

* Why are you working in this field of data and knowledge management?

I am interested in archiving and structuring knowledge using a fine grained approach, especially knowledge on how to make things in a sustainable way as proposed by Buckminster Fuller's "Design Science" and many others, and making that knowledge freely available to all people of the world to help reduce ignorance and want and to increase humanity's chances of surviving with style on Earth and even someday in space habitats. See: http://www.kurtz-fernhout.com/oscomak/index.htm for more details. If you are interested in others doing work with knowledge about sustainability, look at: http://www.bfi.org/ and: http://www.humaninfo.org/

* Anything else to know?

Pointrel is a trademark of Paul D. Fernhout.