versăo $Revision: 1.2 $, November 2000, in Stockholm
This document continuously presents the state of the Makumba project. It is intended for Makumba designers and implementors. The idea is to keep all knowledge about Makumba in one place and not to have it spread in all sorts of files. The spec will be maintained in parallel. As different from spec, this document also includes past ideas and future visions.
The only information that is not in this document are the introdutory text, the spec, the API documentation, the tasks, and the supported databases documentation. (need linked summary at the beginning and number-indepedent A NAMEs. automatic section numbering seems impossible in Composer)
(see Cristi's Philosophy mail)
pre-packaged BEST features are very different from professional
- no fine-grained control of all details
- a feature is available on the most generic sense with no concern for speed or
space
- few features are available, only the essentials
- makes it easy to learn and develop
- not simple-to-complex learning because an advanced user might use complex
features that a novice won't understand
non-professional
- total control
- feeling of independence
- no general purpose (taught in school) commercial technology exists for
data-driven web apps (hack over database systems to get to the web or vice
versa)
next (visionary) system will come by design refactoring of the present one
we didn't manage to run Lasso server on the laptop, but we
could look at language examples
LDML looks very much like what we want for Makumba
observations learned from Lasso:
- one can have more than one presentation language
- any language can have an XML description (whatever the use)
- html editors can be used to edit a good language, and if the tags are like
<..> the html editors will not rewrite them (see difference between LDML
and object LDML
- LDML is very much programming (rather than presentation) oriented
- too many tags
- nice simple logging mechanism, but it shouldn't be at presentation layer, or
should it?
- lasso seems to be a one-table search thing
- but this is not a problem since you can have views for each join you can
think of
- SQL dialect dependent
- but more search power
- named queries look interesting
Looks like JML/JSP. Just a different syntax and probably different compilation and error messages. But that doesn't make it much better.
both CFML and LDML miss the generation of the default presentation because they don't describe the data anywhere.
an improvement of CFML and LDML over stuff like PHP or JSP is that there is better integration with the database and less db connection installation overhead
both have server-side includes
Tango (pervasive.com) is a dead product but it makes a nice distinction (presentation, data model and business logic)
Makumba should be oriented in helping trainees to learn it
fast, install it easily and start coding with as few problems as possible
- removing installation overhead
- tomcat, jdk, database engine, etc., should be pre installed and packaged
- MySQL as database for windows
- CD or zip with the pre installation and example data (e.g. complete general
tables)
- CVS as overhead during trainee startup (then very useful)
- we don't necessarily want/have the skills from open-source. Not all trainees
need help there, but it's a good diversification of opportunity.
Igor case: hindrances of the BEST development environment
- a lot of good will
- poor connection
- different locations, slow email communication
- not yet enough experience
- initially a victory (odbc4 driver)
- when the new problem came in (message board), local devel stopped
Student organisation identification with an ideal
- professional software devel (the spec document, framework that was promising
to comply with changes in spec, etc)
- open source (Makumba), no release yet because of prestige. Not all of us have
to be open source programmers. Some trainees will only get to do Makumba code
development and maintenance, some others will get to work on spec and internals
in the open-source process.
(kosice) meeting preparation with hands-on examples (low overhead should help this)
(this is working information for now, but might be later "demoted" as an annex)
- abstract, DB and presentation data description
- only had presentation implemented
- re-organised directories in package like manner
- some "redundant" features: templates, one-to-many and many-to-many
tables. redundant features complicate the language while they can actually be
implemented with the existing ones
- we were still puzzled about search
- we believed that there is a ideal presentation layer and we tried to find it
- what we still like: general idea of record description, replication fields,
record title, pointer, set, general type correspondence, FIXED, NOTNULL
- implemented the first version based on these ideas
- proposed searches in separate files. still not clear what
to do with arguments
- still not clear how to select fields from the search
- attempt a second presentation layer; quite cryptic. See Annex 1
- "new database layer" attempted to model all SQL SELECT concepts in
Java objects (Expression and Filter)
- presentation toolkit and other upfront design OOP ideas, for generating any
kind of content, etc...
- ODMG database layer proposed OODB-like principles for the API
- JSP taglib-based presentation layer. See Annex 2
- OQL discussion - resulted in Autumn 2000 implementation of OQL
we liked the join in the FROM. it "moves" the join from the
WHERE
clause, which is mostly for filtering
SELECT DISTINCT s FROM /best/student s, s.education e,e.studyfield st
WHERE st.name="Chemestry"
joins in the implicit relations?
/best/minerva/student.minerva m(s)
- attributes are kind of constraints
- not table, but object-oriented
- lists as a clear difference from sets
- inheritance rather than inclusion
- multilanguage suport in text literals
- title is the first non_pointer field
- clean up of syntax
- business logic
- actors
- authentication mechanism
- "extension"
- administration mechanism
- new field constraints UNIQUE, ENCRYPTED
- minimum and maximum char length [a..b]
- if we define inline types, we don't have to think of
complex sets and 1:1 pointers anymore. this simplifies the language
- Toto's Business Logic sources.
- fieldname:
int { default longhish_shit_that_is_not_comfortable_to_repeat, a, b, c }
- now as default for date
- interfaceLanguage in actor profile
- %localize(lang1, lang2) at presentation level
- dot instead of / in the class name. capital for type name as style rule. OQL
becomes:
SELECT DISTINCT s FROM best.Student s, s.education e, e.studyfield st
WHERE
st.name="Chemistry"
- data ecryption not one way
- spec shouldn't talk about SQL, or specify that it is an example
- tried to make sense of all stuff and set short-term and
long-term goals
- see design issues below, plus annexes 4 and 5
- we ended in a frustrated note due to too short meeting time. But overal the
meeting was very productive.
- ODBC and mSQL (temporarily) not supported
- Implemented OQL (with antlr, fixed their
OQL grammar, submitted it, got no reply yet)
- bought makumba.org domain name
- made a database copy utility (a good ground for future replication, transfer
between different databases, different SQL servers, cleaning historical table
names)
- moved some Toto's own methods (presumably most frequently used by a app
developer) to a prominent place (metadata.db.Database) for everybody to use
- webmasters looking for alternatives (PHP, Perl, Apache server side
includes...)
- subselects in MySQL not fully supported
- realised that database-level indexing improves performance significantly
- made OQL work in minerva student search
- compiled and packaged a set of programs (17 megs) needed to run an
application (in this case Minerva, working on a non-programers' laptop) No need
to set any PATH or CLASSPATH. Only needed change might be to enlarge the memory
of the command.com but we can provide proactive assistance for that
- confronted webmasters with jsp --> enthusiasm!
- made BEST realize of Makumba's importance to the organisation / raised the
interest by introducing Karamba!
- markeTeam takes over the PR campaign- promotion of Makumba as an open
source project
- eliminated 'dot convention' - no more dots in table names
-
eliminated the need to manually translate names of fields and
tables to avoid conflicts with SQL keywords on some database engines (perticularly
mySql is very sensitive, postgres more tollerant)
- added a feature to shorten table names (Postgres has limit to 32 characters,
MySql no problems)
- documentation clean-up
- updating this doc, API spec
- found a nice model for Rapid Application Development
- First test of a bigger application with the latest version
of Makumba.
- Presentation to webmasters, they love it, we need to make them a distribution
for them to play with it.
- Writing the Makumba
JSP tag specification, finalizing the taglibs.
- New Makumba DD language implementation postponed (and have it done later with
antlr)
The objective was to get to the minimum powerful makumba to be able to implement Karamba
Solved database config files
Decided to implement types in pointers
Replication when DD changes. What happens to the LBG copies? How to detect DD and logical changes?
Dillemas about Archive and Document. We didn’t know how to organize the DDs, and how to set the access rights.
Dillemas about static (database registered) and dynamic (application dictated) access rights
Application template, components (maybe configurable by webmaster), maybe new tags, skins around them
DD-parallel code, express there all we wanted to express in the DD Business Language for now. Such classes will be called by the taglib directly (no more JSP parallel code). If a class doesn’t exist, a default one is used. If it exists, it is a sublcass of a powerful toolbox (refactored from needs and experience) class that knows
q rigts set (static, or by OQL)
q decorated (pointer) field attribute
q unique field attribute
q !unique constraint OQL expression
q computed fields
q unique and notnull to implement directly in SQL
(cristi toilet) 3 tier application: servlet container, DD-parallel code, database. Will improve development speed
q JSP calls DD-parallel methods of the kind: “do(type, operation, data) in the main tables
q DD-parallel code calls methods of the same kind in the tables and subtables, after security checks, etc.
We realised that insertUnique just comes from the unimplemented unique constraint
We have a clear idea of what an application is. Application come in bean-like components (maybe will be implemented as java beans?)
- DDs, with a field that indicate which instance of the app do we talk about (name)
- DD-parallel files, written in general
- JSPs, maybe with an argument
- Maybe a special tag, say <karamba:todolist specificArguments../>
package name metadata –> org.makumba
Issues are divided in current problems (5.1), short term objectives (5.2) and long term guidelines (5.3). Every such timeframe is divided in General principles, Data description, BEST (organisational/Business) Logic (BL), Java API, Presentation and Development support. Development support describes objectives for fast development and change by experts, trainees, webmasters and normal users.
The new spec will be written by Stefan and comply to 5.1 and 5.2
The long term spec will be developped based on 5.3
as a testbed for new taglibs and API we can use few pages from minerva, see how it feels, what webmasters feel about it.
ANT distribution
get the business logic in DD-parallel code
finish makumba!!!
unique, unique constraints, list, fixed, default values…
specify and implement DD-parallel code
-
make use of strong context provided in the data description
files and in the presentation. Making use of context leads to less
initialization code, less installation overhead
- API growth by pattern discovery. When typical features are identified, they
are included in the toolbox
- decoration will come by refactoring
- authentication: !auth=field_name. the class of the field has to have at least one authenticate* method
- shouldn't the authentication field be always UNIQUE, FIXED, NOTNULL?
- shouldn't !admin be a method, like !title and maybe other commands... ?
- an actor has an authentication mechanism or is a decoration of it
-
authenticating the extension just needs to authenticate the
base
!base=class_name
- how to express in the language that 2 objects of different classes have the same auth?
implement the Database config files with names like
localhost_mysql_makumba
malaka.best.eu.org_pgsql_minerva2
table alteration for the whole DB
There is no way of setting a field to null. There should be a Null value defined at the abstract level.
implement database catalog and full database copying
subtables (ptrOne, set*) should accept data in insert and return data in read. Data should be part of the Dictionary for ptrOne and Enumeration for sets. This looks like it can be written at the metadata.db.level
delete with deletion of internal sets and pointers...
list, set, bag API
profiling of the code once everything is implemented
change /best/student -> best.Student, relative:type from the same package in reference and include
transaction
text type
notification services
DBSV autogeneration
keep configuration in a special table
database copying starting from a type
implement title as first non-pointer field
implement full-text search
parse default values
implement LIST, FIXED, NOTNULL, UNIQUE
Database bugs:
Other db problems
- find other Windows development DB engine (recompile Postgres with cygwin32 ?)
metadata.abstr.printer.subtablePrinter prints the description after all the fields of the subtable. should be before
finish taglib
implement RAD interface development
existing: support for pre-Makumba existing tables: changes of table and field names. nice stuff
- the present Makumba code can be kept (taglibs, tables and
searches can coexist or be based on it)
- maybe logging
- non-pointer inclusion of types
- - a well-formatted metadata file should somehow mention its name for easy citation
- classes rather than tables. objects rather than table records. this needs discussions... instances of applications, etc. See OODB ideas (Annex 6)
- Constraints, field constraints, sets are a constrained bag
- field seperator in DD files (newline too strict?)
- default values
- title as expression rather than field
- if you mention an decorator object (say minerva.student) and the base object exists, the mentioned object is automatically created.
- if the base object doesn't exist, it is created first.
- editing/deleting the base object can be a default right?
- should based types be returned together with the base ?
- should the base object be somehow notified when a wrapper is created?
- Toto: Can it be recursive (i.e. decoration of decoration) ? With the current definition, I don't think so. Moreover we don't really need it, and it may be more difficult to handle...
- Toto: Can an extension be authenticated in a different way than its base ? (i.e. !authenticate)
- Toto: (see annex 3) Does /best/minerva/student inherit the !admin from /best/student implicitely (i.e. Helpdesk) ? I would say yes. And in case !admin is defined in /best/minerva/student ? Should it be the union of admins of /best/student and /best/minerva/student or only admins of /best/minerva/student ?
- virtual sets with arguments
- virtual set with arguments == search?
- virtual set called from another virtual set
- subsets of (virtual) sets, lists of (virtual) sets, etc. introduce constraints beautifully
- virtual set inside the virtual set (see johnny student)
- static virtual set
- wrapper managers are a more general concept, should move up from database. Should be metadata.abstr.WrapperManager or so
Abstract level bugs
- if there is a redirection to a type which is redefined on a lower family, the
lower family type should be obtained. Instead, the same family type is obtained
now
Save OQL searches for future use (with different parameters). Maybe keep them as stored procedures
Database bugs
OQL
q if just a label is mentioned in where/select then the primary key field of that table is used
q parameters $number
q NPE bug if no label and no type exists, a "best" table is mistakenly considered
q best.Student is considered, should be error... or is this windows?
q unmentioned labels should not generate joins
q multiple objects &expressions in SELECT
q expressions in select and where are equivalent, so it's easy to have joins generated there
q subselects can be implemented, inheriting the labels of their containing selects. how about the SQLs that don't support subselects?
- SQL verbosity level
- make sure that webmaster can't change the access rights of
a page
- support modification in html editors?
- not much added for programmers (maybe the new generated classes best.Student().insert(data))
Design will continue from here.
- data description modification: try to detect and signal
all presentation files and java files that might be affected
- java code in the data-description file, or modification of generated java
file out of the description?
- problems with modification of original data description when default code
generation is used at presentation and data level (and then we let the
programmer/webmaster alter the code). Toto proposes class inheritance for the
generated Java files (you extend the generated file, which stays untouched)
- dealing with data description changes: there will be extensive change
detection (i.e. analysis that takes into account the previous version) and
automatic fix of database tables, database code and presentation code (this
supports automatic preservation of existing data and code)
- types of data structure change: field name change, field type change, field
move in another table, ptr -> set transformation (and back?)
- transactions, table and record locking
- logging. what level? actor? db?
- XML dump and import. DTD
- data travels in XML rather than Hashtables
- allow the server to receive data described by a certain DD via email
- support for backup and versioning at db level
- multilingual support tables, enumerated type string, default values (mb also
presentation layer)
- attachement conversion... what level? presentation? preprocessing and save as
text or html??
- replication... keep date of last replication. transitivity. how does the data
structure change support affect replication? quite much... replication of data
descriptions between sites
- selective replication: a location could get replicated locally data that is
more related to its people
- support again more DBs (ODBC, msql, etc)
- looks like we can't count on ODBC (LIKE, close-on-exit, different versions...).
on the other hand, it's cool to be able to generate excel views of the data. we
could support it partially.
- remote database??? Old description: to support remote access to a database,
we need:
- need notification services for insert and update.
different actors may choose to be notified when a new record is inserted or a
certain record is changed. only some such records may be relevant for the
respective actor, so filters may need be installed. access rights problems
exist here (i.e. I may be notified only of records I am allowed to see)
- should pointers be represented as 1 or as 2 SQL longs (java ints)
- save search parameters for future refinement and use…
- relations with other buzz technologies: Java Beans, WAP
- change parser to understand ENCRYPTED. field encryption
not one way
- surely implement on-the-spot types (to replace ptrOne and setComplex)
- omit ptr keyword
- minimum and maximum char length [a..b]
- fieldname: int { default longhish_shit_that_is_not_comfortable_to_repeat, a,
b, c }
- now as default for date
- database-platform independence (compilation up to stored procedures for
higher performance)
-
actors, rights and other constraints, erros if constraints are
not respected, actions (field sets, etc)
who : action (fields) precondition, operations, postcondition
- actions (understood as method adjustments) are actually expressions of DB redundancies? Toto: constraints can also be expressed in actions
- the meaning of set in argument lists and method results seems to be different from the meaning of set in sets and virtual sets
- implicit assigments might be difficult to "read"
- methods with no actor cannot be called from presentation
- static methods
- look again into best/johnny/student (see annex 2)
- looks like constraints don't have to sit in the method. but then, how do you ensure that a consistent state is kept?
- we could say "these 2 op form an atom. don't check consistency between"
- how to name decorator base when used in argument lists?
- should loose importance but should still exist
- ODMG, OODB ideas
- optimize current drivers
-
consider also introducing relationships on the database level
- Stored procedures from OQL or CREATE VIEW ?
- evaluation of the select() should be lazy. That is, the SELECT should be
executed only when the hasMoreElements() or nextElement() methods of the
returned enum are called
- there could be a form of Table.insert() that gets a record set (not just a
record) as argument. At SQL level this could generate an INSERT
INTO table SELECT ... if the recordset comes from a search
_in_the_same_database_
DBSV auto generation?
support for existing tables: besides changes of table and field names, import structure from SQL tables, changes of type, automatic change of #alter file, facilities of changing types (such as a abstract-level int{..=1 ...=0} that might actually be a BIT in the existing pre-Makumba DB)
it's not clear what will happen with DBSV for an existing database. A possible solution is to consider it 0 for start (replication is not on anyway). Before the first replication, a pointer transformer can be executed on each table, to adjust all pointer fields
- redirection property files could be eliminated. each record handler should have an associated redirector class, that has methods for each redirected type, and a getFamillyName method. Due to the getFieldMethods(), each handler family has to have a FieldHandler (i.e. cannot inherit it). That's wrong (e.g. metadata.sql.odbc and metadata.sql.msql don't need FieldManagers, they are just empty extensions of metadata.sql.FieldManager)
- XSL is not webmaster-friendly, but XML+XSL= presentation.
Maybe XSL should be generated out of our own presentation language
- we don't want access rights (field level and actor) in the presentation
- database-to-presentation tools that provide default views can be found in:
- IDEs
- database management applications (Access, Oracle 8i)
we should look at those, just the same as we looked at LDML and CFML
BUT none of these are easily accessible to the BEST webmaster and BEST
programmer on the BEST platform and server in the distributed BEST environment.
any of them can change and the system should survive
- need to check enhydra.org http://www.enhydra.org/software/enhydra/index.html
- support modification in html editors
- interfaceLanguage in actor profile. how should it work?
- how should the content in the web pages be localized?
- %localize(lang1, lang2) at pres level
- think of other kinds of presentations: AWT, RTF, XML, WAP, etc
- the moment you have the profile of an action, you can make
a default JSP page that uses it. that page already has access rights on it
- installation: one can automatically install ODBC data sources with
odbcconf.exe
- (html and view source) is an epitome of low installation and initialisation
overhead
- provide for trial and error experimentation
- maybe we need to look a bit of how webmasters work (see the alttabing stuff
in the cfml tutorial)
- pay attention to errors, where should they come up and how. should not be
hidden in the source
- manual, reference manual, functional sample applications (FAQ, support forum,
discussion, to-do, calendar, system db, bug db, see mail about small app ideas)
- wizard (sounds too professional)
1 new metadata file
2 links to generated presentation and business logic
file
3 the generated files contain comments with
explanations, suggestions and links to tutorial. the form and comments of
generated code can be adjusted by setting a "level of expertise"
4 the programmer can edit the default presentation and
business logic
- in the general case, altering should be done by ALTER TABLE, though a
convenience java alter should be provided
- GPL needs be checked in connection with the antlr OQL grammmar we use
Thanks to BEST for trusting us to come up with something better than Lotus Notes.
Thanks to LBGs in Athens (summer 1999), Chania (autumn 1999), Kosice (summer 2000) and Coimbra (autumn 2000) for their kind hosting support.
Thanks to Yngve for supporting our web-programming ideas (since JML in 1996) which were not on the research agenda and even employing supplementary personnel for their development.
Annexes present code snippets that might still be useful for future development
Date: Thu, 31 Aug 2000 19:05:26 +0200
From: Cristian Bogdan <cristi AT nada.kth.se>
1. common technology:
- our data is fragmented across more technologies. In Notes, data is fragmented
across more databases, because we don't master the technology or simply because
Notes sucks.
- we need total control over the technology or a very good customer service
from whoever provides it
- in order to achieve a low learning (and maintenance, if we do it) treshold,
it has to be simple, i.e. have as few features as possible, the minimum
necessary for us
- it should be portable to any platform our low-budget needs will get us to
2. what is the nature of our applications?
- we have web database applications.
- thus the technology should have ready-made building blocks oriented to fast
generation of such applications
- it should also provide the power for webmasters to adjust the pages to the
lowest detail
- same goes for programmers
3. how do our applications evolve?
- we rarely do ready-made applications. normally we have the minimum necessary
to gather some data. this ensures early availability of a working application,
without much efforts from or pressure on the very-specialised-people (this
corresponds to data description files)
- then (when programmers have time or pressure) we add more and more logic, or
provide ways to enter more data. (this corresponds to business logic level)
- in the meantime, webmasters (when they have time or when they are pressed)
add more and more layout (this corresponds to presentation level)
4. problems with commercial technologies
- they usually contain loads of functions to make for the needs of all big
customers
- ... and to obtain lots of speed for very specialised tasks
- if the technology is commercial we should be able to obtain not only the
technology itself but also its customer support and make sure we can provide
for its maintenance overhead
- commercial would usually tie us to a vendor. BEST breaking relations with the
vendor results in problems. ITC starting to hate the vendor gives other problems
- commercial might tie to a platform or a set of platforms and porting to a new
platform is independent on us
5. "external" technologies (open source or commercial):
- they are used on more than one place so they can be more reliable
- there is a dedicated maintenance crew
- if we can't find any external, we can get to both these advantages by
i) making a good design
ii) implementing it
iii) open-sourcing and publishing it in good places (i.e. places we looked for
one and didn't find :)
<h1>All
students:</h1>
$/best/student
[
Make a <a href=$newLink>new</a> student or check this
list:<p>
{
this is a student:<br>
name is $(name)[$value] <b>$(surname)[$value]</b><br>
<font size=-2>
<a href=$editLink>edit</a>
<a href=$deleteLink>delete</a>
</font><br>
$(birthdate) [<font size=-2>$value</font><p>]
$$[<b>$description</b>:$value <br>]
}(<br>)
]
<%
// give a name to the data to be formatted (there might be better ways to do
this...) %>
<%pageContext.setAttribute("std1",
someDictionaryFromTheDatabase);%>
<%// I want
to format a student for editing, this should open a <form> %>
<mtd:record
structure="/best/student"
data="std1"
submitMethod="editMyself"
afterSubmit="index.jsp"
success="<font color=green>Changes done!</font>"
failure="<font color=red>Changes not done:
$error</font>"
>
<%// I want to format some fields, separated by <br> %>
<mtd:fields names="name,surname,birthdate"
separator="<br>">
<%// field description in bold, then :, then value in a form input%>
<b><mtd:fieldDescription/></b>:
<mtd:fieldEditValue/>
</mtd:fields>
<%// now I format other fields, a bit shorter but with the same result, the fields are displayed for reading ($value), not as html form fields ($editValue) %>
<mtd:fields names="military, etc" pattern="<b>$descr</b>:$value" separator="<br>" />
<%//format the addresses for reading %>
<h2>Addresses</h2>
<dl>
<mtd:set name="languages" separator="">
<%// the address kind as a <dt> %>
<dt><b><mtd:fields name="kind"
pattern="$value" /></b>
<%// all unmentioned fields, i.e. all but "kind" , as
<dd>%>
<dd>
<mtd:fields names="$rest" pattern="$name:$value"
separator="<br>"/>
</mtd:set>
</dl>
<%// ok,
that's it for this student, this should close the <form> %>
</mtd:record>
-----------/best/auth------------------
username : UNIQUE
FIXED char[1..40] ; EN=username
password : ENCRYPTED
char[4..40] ; EN=password
code1 : int ;
first 5 digits
code2 : int ; last
5 digits
<authenticate>
<arguments>
username
password
</arguments>
<result>
result :
/best/auth (
result
.username==username
result
.password==password
)
</result>
</authenticate>
<authenticateByCode>
<arguments>
code1
code2
</arguments>
<result>
result :
/best/auth (
result
.code1==code1
result
.code2==code2
)
</result>
</authenticate>
<new>
<arguments>
username
password
</arguments>
<body>
</body>
</new>
<edit>
<actor>
this
</actor>
<arguments>
password
password2 :
char[4..40] ; confirm password
</arguments>
<body>
password==password2
; wrong password
</body>
</edit>
---------/best/student----------
!authenticate=auth
!title=auth.username
auth : UNIQUE FIXED NOTNULL /best/auth
firstname :
char[1..40] ; EN=firstname
lastname :
char[1..40] ; EN=lastname
address : set
{
kind : UNIQUE char
EN={"home addres","study-time address"}
street : char[80]
city : char[80]
counbtry :
/general/country
}
<new>
<actor>
anonymous
</actor>
<arguments>
firstname
lastname
password
password2 :
char[4..40] ; confirm password
</arguments>
<body>
password==password2
; wrong password
auth.new(firstname+"
"+lastname,password)
</body>
</new>
-------------/best/minerva/student-------------
!base=/best/student
!admin=set
/best/minerva/admin
restriction=set /best/minerva/company ; companies not allowed to see my data
<edit>
<actor>
this
</actor>
<arguments>
restriction
base
</arguments>
<body>
</body>
</edit>
<view>
<actor>
set
/best/minerva/company
</actor>
<arguments>
base
</arguments>
<body>
!restriction.contains(actor)
</body>
</view>
-------------/best/minerva/admin-------------
!authenticate=auth
auth : UNIQUE
FIXED NOTNULL /best/auth
-------------/best/minerva/company-----------
!authenticate=set
/best/minerva/company.representative (authenticate.company==this)
name : UNIQUE FIXED char[1..40]
<search>
<actor>
this
</actor>
<arguments>
languages : set
/general/language
interests : set
/general/interest
universities : set
/general/university
</arguments>
<body>
actor.paid() ; You
are not allowed to search for student : pay the entrance fee first...
</body>
<result>
result : set /best/minerva/student
(
!(restriction.contains(this)
result.student.interest.intersection(interests).size>0
result.student.studies.university.inter(universities).size>0
)
</result>
</search>
----------------/best/minerva/company.representative-------------
!authenticate=auth
!admin=set
/best/minerva/admin
auth : UNIQUE
FIXED NOTNULL /best/auth
company : FIXED
NOTNULL /best/minerva/company
----------------/best/johnny/admin-------------------------------
!authenticate=auth
auth : UNIQUE FIXED NOTNULL /best/auth
---------------/best/johnny/student-----------------------------
!admin=/best/johnny/admin
!base=/best/student
season : set
/best/johnny/season (
applic :
set /best/johnny/application (
applic.student==this AND applic.season==season
)
applic.size()>0
)
applications(season
: /best/johnny/season) : set /best/student/application (
applic.student==this AND applic.season==season
)
-------------------/best/johnny/season----------------------
activities : set
/best/johnny/activity (activities.season==this)
announcementDL :
date
applicationDL :
date
rankingDL : date
optimise()
----------------/best/johnny/activity----------------------
applications : set
/best/johnny/application (applications.activity==this)
ranking : list
/best/johnny/application
rankingPlaces :
int ; number of accepted people
season : FIXED
NOTNULL /best/johnny/season
----------------/best/johnny/application------------------
activity : FIXED
NOTNULL /best/johnny/activity
student : FIXED NOTNULL
/best/johnny/student
extraAccepted :
date
season : /best/johnny/season (season==activity.season)
<new>
<actor>
set
/best/johnny/student
</actor>
<arguments>
activity
</arguments>
<body>
season.applicationDL.after.now
; Too early to apply
season.rankingDL.before.now
; Too late to apply
student=actor
student.applications.append(this)
</body>
</new>
---------------------minerva.student.register.jsp------------
<metadata:new
type="/best/student" response="registerResponse.jsp">
<! register
creates the student ; it has a response form...>
--------------------minerva.student.login.jsp-----------------
<metadata:login
type="best/minerva/student">
<! we could
have different kinds of login :
<metadata:login
type="/best/johnny/student" type="http"
method="authenticateByCode">
--------------------minerva.student.index.jsp----------------
<metadata:actor
type="/best/minerva/student" login="login.jsp">
<! by default,
login="login.jsp">
<! if the
/best/student exists but not /best/minerva/student, there should be a
"base-extension" notification (specified in login.jsp)>
<metadata:view>
------------------minerva.company.student.jsp--------------
<metadata:actor
type="/best/minerva/company">
<metadata:retrieveObject
type="/best/minerva/student" name="student">
<! retrieve the
student from the http arguments>
<metadata:view
object="student">
DataDescription generates and compiles JavaFile
ModifiedJavaFile extends JavaFile
Presentation calls ModifiedJavaFile
DataDescription generates Presentation
GeneratedPresentation is insert, edit, delete, view-all pages
GeneratedPresentation is generic -> the only reference to DataDescription is the name
OR: GeneratedPresentation is not generic, it refers DataDescription fields, especially title, pointer and set fields. it is fully commented
Problems:
1. what if somebody wants to use another structure than IEDVA? what if somebody
wants different filenames?
URL is part of the presentation so it can be changed creative
two different pages might do essentially the same IEDVA operation
immediate answers
- tutorial and scrap areas
- default area (if stuff is to be changed it is copied in other areas)
2. What happens to the presentation when DD changes?
professional solution: semantic analysis of Makumba tags in Presentation
low profile solution: identify the pages that refer to the DD and list them
/*
name is the title fields, so we format it differently */
<b>
<mtd:field name="name"></b>
/* we put the
title first, it is the field "name", the first non-pointer field in
the data description, be aware that if you change the DD, you might change the
tilte field*/
<b>
<mtd:title> </b>
novice-expert modes for comment level
/* we could
have done all this by saying <mtd:record>. this would use all the above
defaults */
needs review
Unlike classical relational DB operations (based on insertion, update and deletion into tables), object repositories are based on writing down in the database all changes that occur to the fields of the objects involved in a transaction. Instead of thinking in terms of tables, the programmer can then only consider the objects involved, and the transactions necessary for their state to be consistent.
Metadata already has concepts from object databases, including pointers, sets, etc, and already defines a database scheme, which makes it easy for it to support a more object-like update mechanism.
Metadata can actually be brought closer to Object database management and ODMG standards by:
If APIs look like below, metadata/db can be implemented using any ODMG (or later JDO) implementation (e.g. Sun's Java Blend already implementsODMG, we could try to see if we can't implement metadata/db with it already, there is a 30 day trial).
To get faster access to fields, classes can be generated automatically out of the metadata files. In order not to need JVM changes or class file postprocession, field changes/reads can be accomodated with methods that follow the Java Beans property name patterns: setXXX(Object) and Object getXXX(), where XXX is the field name. This patten can co-exist with the present object-Dictionary approach.
All records are represented in memory as DBObjects (can extend Dictionary). To pass a pointer to an object, one just has to pass that DBObject. Such DBObjects can be registered into the Database (which will lead to an INSERT), and some of them can be given special names. Queries made in only one table return collections of DBObjects, and pointer and set fields will come as DBObjects.
DBObject methods for reading :
Fields are returned as:
Modifications (incl insertion) and destructions are performed inside transactions. Every thread is part of at most one transaction. A thread can join an already existing transaction. If a write operation is invoked, but no transaction is associated with the thread, a transaction consisting of only that operation will be performed.
Write operations will cause one INSERT or UPDATE for each affected object inside a transaction. INSERT is performed on an object with no pointer (and leads to setting that pointer), and UPDATE otherwise.
Object destruction: DBObject has a delete() method. All ptrOnes, setInt, setChar, setComplex will be deleted (in the same transaction).
A transaction can lock certain DBObjects for reading or writing. A transaction can be cancelled (abort, nothing is written to the database), or declared finished (commit, all changes are written). Also, the transaction can finish and re-start immediately, without releasing any locks.
At JDBC level, the Database will employ one Connection per transaction. Connections can of course be recycled.
The database has a cache, having the composite key [RecordInfo, ptr]. The cache contains data from the respective table (eventually in an array Object[] for each record).
The DBObject contains just a reference to the cache entry where its data stays. When the DBObject goes out of scope, the cache is notified that the respective entry has one less usage. A cache cleaner passes regularly and deletes cache entries which are not used.
To achieve this cache cleaning, based on the DBObjects going out of scope, one has to build a DBObject instance whenever a query result is returned, or pointer de-reference is made. That means that two DBObjects can represent the same thing, so changing a DBObject might affect another DBObject. To test if two DBObjects represent the same record in the database we can have a method such as
To avoid this, one has to use Java 2 (JDK 1.2) weak references. Using java 2 will also bring more standardization with the Collection classes. But for that, we need to upgrade to Java Web Server 2.0 (which might solve other problems we have now, such as page recompilation), currently Johnny and Minerva use Java Web Server 1.1.
Locking can be implemented at Java level or at database level.
For many of the classes (Database, Transaction) we could directly implement the ODMG interfaces. This will create a ODMG-like "metadata binding", which will give a standard value to metadata.
OODB discussion (Cristi and Stefan in Technico). This
discussion probably came from the data-code relationship... In normal DBs, all
objects of a type are accessible by a simple select. In OOP, objects are
accessed thru references. This looks strange. Proposal to implement data
inheritance with 2 extra columns (one table name and one pointer) in each table
that points to a record containing the extension columns in separate extension
tables. For example, a committee might request an extension to their archive,
which in principle will behave like all other archives. This leads to thoughts
about application inheritance, etc. In an OODB approach, viewing a table
actually would view one single virtual set.
useful for development
|
BIGINT |
-5 |
|
constructor RecordFormatter(RecordInfo) |
initializes the formatter from a given metadata |
|
void writeInPage(Writer stream, Dictionary data) |
writes a view of the given data to the given stream |
|
void writeInForm(Writer stream, Dictionary data) |
writes an edit form of the given data to the given stream |
|
void writeInFormNew(Writer stream) |
writes an empty form (with default values) to the given stream |
|
Dictionary readFromQueryString(Dictionary query) |
reads the field from the http query |
adjustment methods (less urgent):
later:
|
void writeInFormSearch(Writer stream) |
writes a search form |
|
void writeInFormFormat(Writer stream) |
opens the formatter for editing to a non-tech user, as a edit form |
|
void writeInPageFormat(Writer stream) |
pens the formatter for editing to a non-tech user, as a page |
/best/student/$edit
/best/student/$view
/best/student/$new
/best/student/$list
are a tempting way to go. Then one just has to pass a pointer (for $view and $edit) to get results, which should be the defaul for $list views. A generic servlet interpreting this (reading the path to the metadata from its url path, reading the pointer and the form data from the query string, etc) can handle all cases needed, generating automatically some jhtml/jsp source for each case, which will then be adjusted by the webmaster.
Problems:
/organiser$/best/student/$view
then organiser could be a search in the auth and student tables that would give
a non-empty result if the respective is an organiser
getFormatter().format(getData())
where getFormatter returns the formatter adjusted in the format() method, and format() returns the data identified by the pointer (for $edit and $view) or the bunch of data selected by $list.
/best/student/$view$format
This proposal was made for a fast implementation process and to give a well-known place for the java code.
Every JSP page that uses the Makumba taglibs will create a parallel Java class per operation invoked. The first page execution will call default implementations of the methods in the class and may announce the developer where to go to edit the code. The "pointer" to the code can stay in the generated HTML as a comment. It is intended that when the parallel code changes, the JSP is touched and re-compiled automatically. The JSP will call dedicated callbacks in the parallel class at different moments in the execution as illustrated below. References to the page and to the "brother classes" generated by the page will be available.
For example, a typical edit form will invoke the operations "writeInForm" and "edit" (since there are no response pages) So it will generate two parallel classes. Still, in such a case (presentation-response), one might have a single class dealing with the whole transaction, since the two are logically linked.
Still need to consider
- the webmaster-security of the solution
- the mechanisms by which the objects from the parallel class are created and
referenced
- our last dilemma in Coimbra: should the access rights and field access rights
be expressed in JSP-parallel or in DD-parallel (one java class per Data
description, i.e. Makumba file) java code?
- the clear definition of the methods. Below are some
guidlines, but the final interface will come out per operation (view, view in
form, edit, delete, insert) after some practice.
|
method signature |
semantics |
default implementation |
|
Object getInitialState() |
returns the initial state of an operation implementation. the state info will be passed to all methods of the template method |
returns null |
|
boolean isAuthorised(Pointer auth, Dictionary args, Object state) |
gets a pointer in an auth table detected by some http
way |
if you are in the JSP root, return true |
|
Object getData(Pointer auth, Dictionary args, Object state) |
gets the data to view, view in form (pre-edit,
pre-insert), insert, or edit |
one http arg is a pointer, data is retrieved from the main table using it |
|
boolean beforeExcecute(Object state) |
Executes code before the operation (insert, delete, edit) is executed. Returns whether the operation should be executed or not. |
empty |
|
void afterExecute(Object state) |
similar to above |
empty |
|
boolean needTransaction(Object state) |
tells whether the operation and the rest should be wrapped in a transaction |
returns false |
|
boolean canView( String field) |
field access level. not clear if it will be here or not. |
returns true |
general/general_table1 (ex all countries, all languages)
/general_table2 ...
organization1/major_table1 (ex best/student)
/major_table2 ...
/application1/table1 ... (ex best/minerva/company)
/table2 ...
/application2/table1 ...
organization2/major_table1 ...
common_application/table1 ... (ex eventcalendar/event)
This is what the Stockholm meeting had in mind for BEST hierarchy. Object orientation and decoration were assumed
Auth
Student
best.Student decorates Student
Group
best.Lbg extends Group
best.Committee extends Group
best.vivaldi.Student decorates Student
best.vivaldi.Season
best.vivaldi.Activity
best.vivaldi.Application
best.minerva.Student decorates Student
Company
best.minerva.Company decorates Company
best.minerva.CompanyRepresentative
best.karamba.Document
best.karamba.Message
best.karamba.Repository
best.karamba.OfficialArchive extends Repository
best.karamba.CommitteeArchive extends Repository
best.karamba.TigroBlabla extends CommitteeArchive
$Header: /usr/local/cvsroot/metadata/doc/makumba.html,v 1.4 2000/11/15 23:10:51 stefan Exp $