versão $Revision: 2.2 $, November 2000, in Stockholm
This document continuously presents the state of the Makumba project. It is intended for Makumba designers and implementors. The idea is to keep all knowledge about Makumba in one place and not to have it spread in all sorts of files. The spec will be maintained in parallel. As different from spec, this document also includes past ideas and future visions.
Recently design issues are kept in Makumba's bugzilla, with designDiscussion milestone.
The only information that is not in this document are the introdutory text, the spec, the API documentation, the tasks, and the supported databases documentation.
(see Cristi's Philosophy
mail)
pre-packaged BEST features are very different from professional
- no fine-grained control of all details
- a feature is available on the most generic sense with no concern
for speed or space
- few features are available, only the essentials
- makes it easy to learn and develop
- not simple-to-complex learning because an advanced user might use
complex features that a novice won't understand
non-professional
- total control
- feeling of independence
- no general purpose (taught in school) commercial technology
exists for
data-driven web apps (hack over database systems to get to the web
or vice versa)
next (visionary) system will come by design refactoring of the
present one
we didn't manage to run Lasso server on the laptop, but we could
look at language examples
LDML looks very much like what we want for Makumba
observations learned from Lasso:
- one can have more than one presentation language
- any language can have an XML description (whatever the use)
- html editors can be used to edit a good language, and if the tags
are like <..> the html editors will not rewrite them (see
difference between LDML and object LDML
- LDML is very much programming (rather than presentation)
oriented
- too many tags
- nice simple logging mechanism, but it shouldn't be at
presentation layer, or should it?
- lasso seems to be a one-table search thing
- but this is not a problem since you can have views for each join
you can think of
- SQL dialect dependent
- but more search power
- named queries look interesting
Looks like JML/ JSP. Just a different syntax and probably different compilation and error messages. But that doesn't make it much better.
both CFML and LDML miss the generation of the default presentation because they don't describe the data anywhere.
an improvement of CFML and LDML over stuff like PHP or JSP is that there is better integration with the database and less db connection installation overhead
both have server-side includes
Tango (pervasive.com) is a dead product but it makes a nice distinction (presentation, data model and business logic)
Makumba should be oriented in helping trainees to learn it fast,
install it easily and start coding with as few problems as
possible
- removing installation overhead
- tomcat, jdk, database engine, etc., should be pre installed and
packaged
- MySQL as database for
windows
- CD or zip with the pre installation and example data (e.g.
complete general tables)
- CVS as overhead during trainee startup (then very useful)
- we don't necessarily want/have the skills from open-source. Not
all trainees need help there, but it's a good diversification of
opportunity.
Igor case: hindrances of the BEST development environment
- a lot of good will
- poor connection
- different locations, slow email communication
- not yet enough experience
- initially a victory (odbc4 driver)
- when the new problem came in (message board), local devel
stopped
Student organisation identification with an ideal
- professional software devel (the spec document, framework that
was promising to comply with changes in spec, etc)
- open source (Makumba), no release yet because of prestige. Not
all of us have to be open source programmers. Some trainees will
only get to do Makumba code development and maintenance, some
others will get to work on spec and internals in the open-source
process.
(kosice) meeting preparation with hands-on examples (low overhead should help this)
(this is working information for now, but might be later "demoted" as an annex)
- abstract, DB and presentation data description
- only had presentation implemented
- re-organised directories in package like manner
- some "redundant" features: templates, one-to-many and
many-to-many tables. redundant features complicate the language
while they can actually be implemented with the existing ones
- we were still puzzled about search
- we believed that there is a ideal presentation layer and we tried
to find it
- what we still like: general idea of record description,
replication fields, record title, pointer, set, general type
correspondence, FIXED, NOTNULL
- implemented the first version based on these ideas
- proposed searches in separate files. still not clear what to
do with arguments
- still not clear how to select fields from the search
- attempt a second presentation layer; quite cryptic. See Annex
1
- "new database layer" attempted to model all SQL SELECT concepts
in Java objects (Expression and Filter)
- presentation toolkit and other upfront design OOP ideas, for
generating any kind of content, etc...
- ODMG database layer proposed OODB-like principles for the API
- JSP taglib-based presentation layer. See Annex 2
- OQL discussion - resulted in Autumn 2000 implementation of
OQL
we liked the join in the FROM. it "moves" the join from the
WHERE
clause, which is mostly for filtering
SELECT DISTINCT s FROM /best/student s, s.education e,e.studyfield st
WHERE st.name="Chemestry"
joins in the implicit relations?
/best/minerva/student.minerva m(s)
- attributes are kind of constraints
- not table, but object-oriented
- lists as a clear difference from sets
- inheritance rather than inclusion
- multilanguage suport in text literals
- title is the first non_pointer field
- clean up of syntax
- business logic
- actors
- authentication mechanism
- "extension"
- administration mechanism
- new field constraints UNIQUE, ENCRYPTED
- minimum and maximum char length [a..b]
- if we define inline types, we don't have to think of complex
sets and 1:1 pointers anymore. this simplifies the language
- Toto's Business Logic sources.
- fieldname: int { default
longhish_shit_that_is_not_comfortable_to_repeat, a, b, c }
- now as default for date
- interfaceLanguage in actor profile
- %localize(lang1, lang2) at presentation level
- dot instead of / in the class name. capital for type name as
style rule. OQL becomes:
SELECT DISTINCT s FROM best.Student s, s.education e,
e.studyfield st
WHERE st.name="Chemistry"
- data ecryption not one way
- spec shouldn't talk about SQL, or specify that it is an
example
- tried to make sense of all stuff and set short-term and
long-term goals
- see design issues below, plus annexes 4 and 5
- we ended in a frustrated note due to too short meeting time. But
overal the meeting was very productive.
- ODBC and mSQL (temporarily) not supported
- Implemented OQL (with antlr,
fixed their OQL grammar, submitted it, got no reply yet)
- bought makumba.org domain name
- made a database copy utility (a good ground for future
replication, transfer between different databases, different SQL
servers, cleaning historical table names)
- moved some Toto's own methods (presumably most frequently used by
a app developer) to a prominent place (metadata.db.Database) for
everybody to use
- webmasters looking for alternatives (PHP, Perl, Apache server
side includes...)
- subselects in MySQL not fully supported
- realised that database-level indexing improves performance
significantly
- made OQL work in minerva student search
- compiled and packaged a set of programs (17 megs) needed to run
an application (in this case Minerva, working on a non-programers'
laptop) No need to set any PATH or CLASSPATH. Only needed change
might be to enlarge the memory of the command.com but we can
provide proactive assistance for that
- confronted webmasters with jsp --> enthusiasm!
- made BEST realize of Makumba's importance to the organisation /
raised the interest by introducing Karamba!
- markeTeam takes over the PR campaign- promotion of Makumba as an
open source project
- eliminated 'dot convention' - no more dots in table names
- eliminated the need to manually translate names of fields and
tables to avoid conflicts with SQL keywords on some database
engines (perticularly mySql is very sensitive, postgres more
tollerant)
- added a feature to shorten table names (Postgres has limit to 32
characters, MySql no problems)
- documentation clean-up
- updating this doc, API spec
- found a nice model for Rapid Application Development
- First test of a bigger application with the latest version of
Makumba.
- Presentation to webmasters, they love it, we need to make them a
distribution for them to play with it.
- Writing the Makumba JSP tag
specification, finalizing the taglibs.
- New Makumba DD language implementation postponed (and have it
done later with antlr)
The objective was to get to the minimum powerful makumba to be able to implement Karamba
Solved database config files
Decided to implement types in pointers
Replication when DD changes. What happens to the LBG copies? How to detect DD and logical changes?
Dillemas about Archive and Document. We didn’t know how to organize the DDs, and how to set the access rights.
Dillemas about static (database registered) and dynamic (application dictated) access rights
Application template, components (maybe configurable by webmaster), maybe new tags, skins around them
DD-parallel code, express there all we wanted to express in the DD Business Language for now. Such classes will be called by the taglib directly (no more JSP parallel code). If a class doesn’t exist, a default one is used. If it exists, it is a sublcass of a powerful toolbox (refactored from needs and experience) class that knows
- rigts set (static, or by OQL)
- decorated (pointer) field attribute
- unique field attribute
- !unique constraint OQL expression
- computed fields
- unique and notnull to implement directly in SQL
(cristi toilet) 3 tier application: servlet container, DD-parallel code, database. Will improve development speed
- JSP calls DD-parallel methods of the kind: “do(type, operation, data) in the main tables
- DD-parallel code calls methods of the same kind in the tables and subtables, after security checks, etc.
We realised that insertUnique just comes from the unimplemented unique constraint
We have a clear idea of what an application is. Application come in bean-like components (maybe will be implemented as java beans?)
- DDs, with a field that indicate which instance of the app do we talk about (name)
- DD-parallel files, written in general
- JSPs, maybe with an argument
- Maybe a special tag, say <karamba:todolist specificArguments../>
package name metadata –> org.makumba
Issues are divided in current problems (5.1), short term objectives (5.2) and long term guidelines (5.3). Every such timeframe is divided in General principles, Data description, BEST (organisational/Business) Logic (BL), Java API, Presentation and Development support. Development support describes objectives for fast development and change by experts, trainees, webmasters and normal users.
The new spec will be written by Stefan and comply to 5.1 and
5.2
The long term spec will be developped based on 5.3
as a testbed for new taglibs and API we can use few pages from minerva, see how it feels, what webmasters feel about it.
ANT distribution
get the business logic in DD-parallel code
finish makumba!!!
unique, unique constraints, list, fixed, default values…
specify and implement DD-parallel code
- make use of strong context provided in the data description
files and in the presentation. Making use of context leads to less
initialization code, less installation overhead
- API growth by pattern discovery. When typical features are
identified, they are included in the toolbox
- decoration will come by refactoring
- authentication: !auth=field_name. the class of the field has to have at least one authenticate* method
- shouldn't the authentication field be always UNIQUE, FIXED, NOTNULL?
- shouldn't !admin be a method, like !title and maybe other commands... ?
- an actor has an authentication mechanism or is a decoration of it
- authenticating the extension just needs to authenticate the
base
!base=class_name
- how to express in the language that 2 objects of different classes have the same auth?
implement the Database config files with names like
localhost_mysql_makumba
malaka.best.eu.org_pgsql_minerva2
table alteration for the whole DB
There is no way of setting a field to null. There should be a Null value defined at the abstract level.
implement database catalog and full database copying
subtables (ptrOne, set*) should accept data in insert and return data in read. Data should be part of the Dictionary for ptrOne and Enumeration for sets. This looks like it can be written at the metadata.db.level
delete with deletion of internal sets and pointers...
list, set, bag API
profiling of the code once everything is implemented
change /best/student -> best.Student, relative:type from the same package in reference and include
transaction
text type
notification services
DBSV autogeneration
keep configuration in a special table
database copying starting from a type
implement title as first non-pointer field
implement full-text search
parse default values
implement LIST, FIXED, NOTNULL, UNIQUE
Database bugs:
Other db problems
- find other Windows development DB engine (recompile Postgres with
cygwin32 ?)
metadata.abstr.printer.subtablePrinter prints the description after all the fields of the subtable. should be before
finish taglib
implement RAD interface development
existing: support for pre-Makumba existing tables: changes of table and field names. nice stuff
- the present Makumba code can be kept (taglibs, tables and
searches can coexist or be based on it)
- maybe logging
- non-pointer inclusion of types
- a well-formatted metadata file should somehow mention its name for easy citation
- classes rather than tables. objects rather than table records. this needs discussions... instances of applications, etc. See OODB ideas (Annex 6)
- Constraints, field constraints, sets are a constrained bag
- field seperator in DD files (newline too strict?)
- default values
- title as expression rather than field
- if you mention an decorator object (say minerva.student) and the base object exists, the mentioned object is automatically created.
- if the base object doesn't exist, it is created first.
- editing/deleting the base object can be a default right?
- should based types be returned together with the base ?
- should the base object be somehow notified when a wrapper is created?
- Toto: Can it be recursive (i.e. decoration of decoration) ? With the current definition, I don't think so. Moreover we don't really need it, and it may be more difficult to handle...
- Toto: Can an extension be authenticated in a different way than its base ? (i.e. !authenticate)
- Toto: (see annex 3) Does /best/minerva/student inherit the !admin from /best/student implicitely (i.e. Helpdesk) ? I would say yes. And in case !admin is defined in /best/minerva/student ? Should it be the union of admins of /best/student and /best/minerva/student or only admins of /best/minerva/student ?
- virtual sets with arguments
- virtual set with arguments == search?
- virtual set called from another virtual set
- subsets of (virtual) sets, lists of (virtual) sets, etc. introduce constraints beautifully
- virtual set inside the virtual set (see johnny student)
- static virtual set
- wrapper managers are a more general concept, should move up from database. Should be metadata.abstr.WrapperManager or so
Abstract level bugs
- if there is a redirection to a type which is redefined on a lower
family, the lower family type should be obtained. Instead, the same
family type is obtained now
Save OQL searches for future use (with different parameters). Maybe keep them as stored procedures
Database bugs
OQL
- if just a label is mentioned in where/select then the primary key field of that table is used
- parameters $number
- NPE bug if no label and no type exists, a "best" table is mistakenly considered
- best.Student is considered, should be error... or is this windows?
- unmentioned labels should not generate joins
- multiple objects &expressions in SELECT
- expressions in select and where are equivalent, so it's easy to have joins generated there
- subselects can be implemented, inheriting the labels of their containing selects. how about the SQLs that don't support subselects?
- SQL verbosity level
- make sure that webmaster can't change the access rights of a
page
- support modification in html editors?
- not much added for programmers (maybe the new generated classes best.Student().insert(data))
Design will continue from here.
- data description modification: try to detect and signal all
presentation files and java files that might be affected
- java code in the data-description file, or modification of
generated java file out of the description?
- problems with modification of original data description when
default code generation is used at presentation and data level (and
then we let the programmer/webmaster alter the code). Toto proposes
class inheritance for the generated Java files (you extend the
generated file, which stays untouched)
- dealing with data description changes: there will be extensive
change detection (i.e. analysis that takes into account the
previous version) and automatic fix of database tables, database
code and presentation code (this supports automatic preservation of
existing data and code)
- types of data structure change: field name change, field type
change, field move in another table, ptr -> set transformation
(and back?)
- transactions, table and record locking
- logging. what level? actor? db?
- XML dump and import. DTD
- data travels in XML rather than Hashtables
- allow the server to receive data described by a certain DD via
email
- support for backup and versioning at db level
- multilingual support tables, enumerated type string, default
values (mb also presentation layer)
- attachement conversion... what level? presentation? preprocessing
and save as text or html??
- replication... keep date of last replication. transitivity. how
does the data structure change support affect replication? quite
much... replication of data descriptions between sites
- selective replication: a location could get replicated locally
data that is more related to its people
- support again more DBs (ODBC, msql, etc)
- looks like we can't count on ODBC (LIKE, close-on-exit, different
versions...). on the other hand, it's cool to be able to generate
excel views of the data. we could support it partially.
- remote database??? Old description: to support remote access to a
database, we need:
- need notification services for insert and update. different
actors may choose to be notified when a new record is inserted or a
certain record is changed. only some such records may be relevant
for the respective actor, so filters may need be installed. access
rights problems exist here (i.e. I may be notified only of records
I am allowed to see)
- should pointers be represented as 1 or as 2 SQL longs (java
ints)
- save search parameters for future refinement and use…
- relations with other buzz technologies: Java Beans, WAP
- change parser to understand ENCRYPTED. field encryption not
one way
- surely implement on-the-spot types (to replace ptrOne and
setComplex)
- omit ptr keyword
- minimum and maximum char length [a..b]
- fieldname: int { default
longhish_shit_that_is_not_comfortable_to_repeat, a, b, c }
- now as default for date
- database-platform independence (compilation up to stored
procedures for higher performance)
- actors, rights and other constraints, erros if constraints are
not respected, actions (field sets, etc)
who : action (fields) precondition, operations, postcondition
- actions (understood as method adjustments) are actually expressions of DB redundancies? Toto: constraints can also be expressed in actions
- the meaning of set in argument lists and method results seems to be different from the meaning of set in sets and virtual sets
- implicit assigments might be difficult to "read"
- methods with no actor cannot be called from presentation
- static methods
- look again into best/johnny/student (see annex 2)
- looks like constraints don't have to sit in the method. but then, how do you ensure that a consistent state is kept?
- we could say "these 2 op form an atom. don't check consistency between"
- how to name decorator base when used in argument lists?
- should loose importance but should still exist
- ODMG, OODB ideas
- optimize current drivers
- consider also introducing relationships on the database
level
- Stored procedures from OQL or CREATE VIEW ?
- evaluation of the select() should be lazy. That is, the SELECT
should be executed only when the hasMoreElements() or nextElement()
methods of the returned enum are called
- there could be a form of Table.insert() that gets a record set
(not just a record) as argument. At SQL level this could generate
an INSERT INTO table SELECT ... if the recordset comes from a
search _in_the_same_database_
DBSV auto generation?
support for existing tables: besides changes of table and field names, import structure from SQL tables, changes of type, automatic change of #alter file, facilities of changing types (such as a abstract-level int{..=1 ...=0} that might actually be a BIT in the existing pre-Makumba DB)
it's not clear what will happen with DBSV for an existing database. A possible solution is to consider it 0 for start (replication is not on anyway). Before the first replication, a pointer transformer can be executed on each table, to adjust all pointer fields
- redirection property files could be eliminated. each record handler should have an associated redirector class, that has methods for each redirected type, and a getFamillyName method. Due to the getFieldMethods(), each handler family has to have a FieldHandler (i.e. cannot inherit it). That's wrong (e.g. metadata.sql.odbc and metadata.sql.msql don't need FieldManagers, they are just empty extensions of metadata.sql.FieldManager)
- XSL is not webmaster-friendly, but XML+XSL= presentation.
Maybe XSL should be generated out of our own presentation
language
- we don't want access rights (field level and actor) in the
presentation
- database-to-presentation tools that provide default views can be
found in:
- IDEs
- database management applications (Access, Oracle 8i)
we should look at those, just the same as we looked at LDML and
CFML
BUT none of these are easily accessible to the BEST webmaster and
BEST programmer on the BEST platform and server in the distributed
BEST environment. any of them can change and the system should
survive
- need to check enhydra.org
http://www.enhydra.org/software/enhydra/index.html
- support modification in html editors
- interfaceLanguage in actor profile. how should it work?
- how should the content in the web pages be localized?
- %localize(lang1, lang2) at pres level
- think of other kinds of presentations: AWT, RTF, XML, WAP,
etc
- the moment you have the profile of an action, you can make a
default JSP page that uses it. that page already has access rights
on it
- installation: one can automatically install ODBC data sources
with odbcconf.exe
- (html and view source) is an epitome of low installation and
initialisation overhead
- provide for trial and error experimentation
- maybe we need to look a bit of how webmasters work (see the
alttabing stuff in the cfml tutorial)
- pay attention to errors, where should they come up and how.
should not be hidden in the source
- manual, reference manual, functional sample applications (FAQ,
support forum, discussion, to-do, calendar, system db, bug db, see
mail about small app ideas)
- wizard (sounds too professional)
1 new metadata file
2 links to generated presentation and business logic file
3 the generated files contain comments with explanations,
suggestions and links to tutorial. the form and comments of
generated code can be adjusted by setting a "level of
expertise"
4 the programmer can edit the default presentation and business
logic
- in the general case, altering should be done by ALTER TABLE,
though a convenience java alter should be provided
- GPL needs be checked in connection with the antlr OQL grammmar we use
Thanks to BEST for trusting us to come up with something better than Lotus Notes.
Thanks to LBGs in Athens (summer 1999), Chania (autumn 1999), Kosice (summer 2000) and Coimbra (autumn 2000) for their kind hosting support.
Thanks to Yngve for supporting our web-programming ideas (since JML in 1996) which were not on the research agenda and even employing supplementary personnel for their development.
Annexes present code snippets that might still be useful for future development
Date: Thu, 31 Aug 2000 19:05:26 +0200
From: Cristian Bogdan <cristi AT nada.kth.se>
1. common technology:
- our data is fragmented across more technologies. In Notes, data
is fragmented across more databases, because we don't master the
technology or simply because Notes sucks.
- we need total control over the technology or a very good customer
service from whoever provides it
- in order to achieve a low learning (and maintenance, if we do it)
treshold, it has to be simple, i.e. have as few features as
possible, the minimum necessary for us
- it should be portable to any platform our low-budget needs will
get us to
2. what is the nature of our applications?
- we have web database applications.
- thus the technology should have ready-made building blocks
oriented to fast generation of such applications
- it should also provide the power for webmasters to adjust the
pages to the lowest detail
- same goes for programmers
3. how do our applications evolve?
- we rarely do ready-made applications. normally we have the
minimum necessary to gather some data. this ensures early
availability of a working application, without much efforts from or
pressure on the very-specialised-people (this corresponds to data
description files)
- then (when programmers have time or pressure) we add more and
more logic, or provide ways to enter more data. (this corresponds
to business logic level)
- in the meantime, webmasters (when they have time or when they are
pressed) add more and more layout (this corresponds to presentation
level)
4. problems with commercial technologies
- they usually contain loads of functions to make for the needs of
all big customers
- ... and to obtain lots of speed for very specialised tasks
- if the technology is commercial we should be able to obtain not
only the technology itself but also its customer support and make
sure we can provide for its maintenance overhead
- commercial would usually tie us to a vendor. BEST breaking
relations with the vendor results in problems. ITC starting to hate
the vendor gives other problems
- commercial might tie to a platform or a set of platforms and
porting to a new platform is independent on us
5. "external" technologies (open source or commercial):
- they are used on more than one place so they can be more
reliable
- there is a dedicated maintenance crew
- if we can't find any external, we can get to both these
advantages by
i) making a good design
ii) implementing it
iii) open-sourcing and publishing it in good places (i.e. places we
looked for one and didn't find :)
<h1>All students:</h1>
$/best/student
[
Make a <a href=$newLink>new</a> student or check this list:<p>
{
this is a student:<br>
name is $(name)[$value] <b>$(surname)[$value]</b><br>
<font size=-2>
<a href=$editLink>edit</a>
<a href=$deleteLink>delete</a>
</font><br>
$(birthdate) [<font size=-2>$value</font><p>]
$$[<b>$description</b>:$value <br>]
}(<br>)
]
<% // give a name to the data to be formatted (there might be better ways to do this...) %>
<%pageContext.setAttribute("std1", someDictionaryFromTheDatabase);%>
<%// I want to format a student for editing, this should open a <form> %>
<mtd:record
structure="/best/student"
data="std1"
submitMethod="editMyself"
afterSubmit="index.jsp"
success="<font color=green>Changes done!</font>"
failure="<font color=red>Changes not done: $error</font>"
>
<%// I want to format some fields, separated by <br> %>
<mtd:fields names="name,surname,birthdate" separator="<br>">
<%// field description in bold, then :, then value in a form input%>
<b><mtd:fieldDescription/></b>:
<mtd:fieldEditValue/>
</mtd:fields>
<%// now I format other fields, a bit shorter but with the same result, the fields are displayed for reading ($value), not as html form fields ($editValue) %>
<mtd:fields names="military, etc" pattern="<b>$descr</b>:$value" separator="<br>" />
<%//format the addresses for reading %>
<h2>Addresses</h2>
<dl>
<mtd:set name="languages" separator="">
<%// the address kind as a <dt> %>
<dt><b><mtd:fields name="kind" pattern="$value" /></b>
<%// all unmentioned fields, i.e. all but "kind" , as <dd>%>
<dd>
<mtd:fields names="$rest" pattern="$name:$value" separator="<br>"/>
</mtd:set>
</dl>
<%// ok, that's it for this student, this should close the <form> %>
</mtd:record>
-----------/best/auth------------------
username : UNIQUE FIXED char[1..40] ; EN=username
password : ENCRYPTED char[4..40] ; EN=password
code1 : int ; first 5 digits
code2 : int ; last 5 digits
<authenticate>
<arguments>
username
password
</arguments>
<result>
result : /best/auth (
result .username==username
result .password==password
)
</result>
</authenticate>
<authenticateByCode>
<arguments>
code1
code2
</arguments>
<result>
result : /best/auth (
result .code1==code1
result .code2==code2
)
</result>
</authenticate>
<new>
<arguments>
username
password
</arguments>
<body>
</body>
</new>
<edit>
<actor>
this
</actor>
<arguments>
password
password2 : char[4..40] ; confirm password
</arguments>
<body>
password==password2 ; wrong password
</body>
</edit>
---------/best/student----------
!authenticate=auth
!title=auth.username
auth : UNIQUE FIXED NOTNULL /best/auth
firstname : char[1..40] ; EN=firstname
lastname : char[1..40] ; EN=lastname
address : set
{
kind : UNIQUE char EN={"home addres","study-time address"}
street : char[80]
city : char[80]
counbtry : /general/country
}
<new>
<actor>
anonymous
</actor>
<arguments>
firstname
lastname
password
password2 : char[4..40] ; confirm password
</arguments>
<body>
password==password2 ; wrong password
auth.new(firstname+" "+lastname,password)
</body>
</new>
-------------/best/minerva/student-------------
!base=/best/student
!admin=set /best/minerva/admin
restriction=set /best/minerva/company ; companies not allowed to see my data
<edit>
<actor>
this
</actor>
<arguments>
restriction
base
</arguments>
<body>
</body>
</edit>
<view>
<actor>
set /best/minerva/company
</actor>
<arguments>
base
</arguments>
<body>
!restriction.contains(actor)
</body>
</view>
-------------/best/minerva/admin-------------
!authenticate=auth
auth : UNIQUE FIXED NOTNULL /best/auth
-------------/best/minerva/company-----------
!authenticate=set /best/minerva/company.representative (authenticate.company==this)
name : UNIQUE FIXED char[1..40]
<search>
<actor>
this
</actor>
<arguments>
languages : set /general/language
interests : set /general/interest
universities : set /general/university
</arguments>
<body>
actor.paid() ; You are not allowed to search for student : pay the entrance fee first...
</body>
<result>
result : set /best/minerva/student (
!(restriction.contains(this)
result.student.interest.intersection(interests).size>0
result.student.studies.university.inter(universities).size>0
)
</result>
</search>
----------------/best/minerva/company.representative-------------
!authenticate=auth
!admin=set /best/minerva/admin
auth : UNIQUE FIXED NOTNULL /best/auth
company : FIXED NOTNULL /best/minerva/company
----------------/best/johnny/admin-------------------------------
!authenticate=auth
auth : UNIQUE FIXED NOTNULL /best/auth
---------------/best/johnny/student-----------------------------
!admin=/best/johnny/admin
!base=/best/student
season : set /best/johnny/season (
applic : set /best/johnny/application (
applic.student==this AND applic.season==season
)
applic.size()>0
)
applications(season : /best/johnny/season) : set /best/student/application (
applic.student==this AND applic.season==season
)
-------------------/best/johnny/season----------------------
activities : set /best/johnny/activity (activities.season==this)
announcementDL : date
applicationDL : date
rankingDL : date
optimise()
----------------/best/johnny/activity----------------------
applications : set /best/johnny/application (applications.activity==this)
ranking : list /best/johnny/application
rankingPlaces : int ; number of accepted people
season : FIXED NOTNULL /best/johnny/season
----------------/best/johnny/application------------------
activity : FIXED NOTNULL /best/johnny/activity
student : FIXED NOTNULL /best/johnny/student
extraAccepted : date
season : /best/johnny/season (season==activity.season)
<new>
<actor>
set /best/johnny/student
</actor>
<arguments>
activity
</arguments>
<body>
season.applicationDL.after.now ; Too early to apply
season.rankingDL.before.now ; Too late to apply
student=actor
student.applications.append(this)
</body>
</new>
---------------------minerva.student.register.jsp------------
<metadata:new type="/best/student" response="registerResponse.jsp">
<! register creates the student ; it has a response form...>
--------------------minerva.student.login.jsp-----------------
<metadata:login type="best/minerva/student">
<! we could have different kinds of login :
<metadata:login type="/best/johnny/student" type="http" method="authenticateByCode">
--------------------minerva.student.index.jsp----------------
<metadata:actor type="/best/minerva/student" login="login.jsp">
<! by default, login="login.jsp">
<! if the /best/student exists but not /best/minerva/student, there should be a "base-extension" notification (specified in login.jsp)>
<metadata:view>
------------------minerva.company.student.jsp--------------
<metadata:actor type="/best/minerva/company">
<metadata:retrieveObject type="/best/minerva/student" name="student">
<! retrieve the student from the http arguments>
<metadata:view object="student">
DataDescription generates and compiles JavaFile
ModifiedJavaFile extends JavaFile
Presentation calls ModifiedJavaFile
DataDescription generates Presentation
GeneratedPresentation is insert, edit, delete, view-all pages
GeneratedPresentation is generic -> the only reference to DataDescription is the name
OR: GeneratedPresentation is not generic, it refers DataDescription fields, especially title, pointer and set fields. it is fully commented
Problems:
1. what if somebody wants to use another structure than IEDVA? what
if somebody wants different filenames?
URL is part of the presentation so it can be changed
creative
two different pages might do essentially the same IEDVA
operation
immediate answers
- tutorial and scrap areas
- default area (if stuff is to be changed it is copied in other
areas)
2. What happens to the presentation when DD changes?
professional solution: semantic analysis of Makumba tags in
Presentation
low profile solution: identify the pages that refer to the DD and list them
/* name is the title fields, so we format it differently
*/
<b> <mtd:field name="name"></b>
/* we put the title first, it is the field "name", the first
non-pointer field in the data description, be aware that if you
change the DD, you might change the tilte field*/
<b> <mtd:title> </b>
novice-expert modes for comment level
/* we could have done all this by saying <mtd:record>. this would use all the above defaults */
needs review
Unlike classical relational DB operations (based on insertion, update and deletion into tables), object repositories are based on writing down in the database all changes that occur to the fields of the objects involved in a transaction. Instead of thinking in terms of tables, the programmer can then only consider the objects involved, and the transactions necessary for their state to be consistent.
Metadata already has concepts from object databases, including pointers, sets, etc, and already defines a database scheme, which makes it easy for it to support a more object-like update mechanism.
Metadata can actually be brought closer to Object database management and ODMG standards by:
If APIs look like below, metadata/db can be implemented using any ODMG (or later JDO) implementation (e.g. Sun's Java Blend already implementsODMG, we could try to see if we can't implement metadata/db with it already, there is a 30 day trial).
To get faster access to fields, classes can be generated automatically out of the metadata files. In order not to need JVM changes or class file postprocession, field changes/reads can be accomodated with methods that follow the Java Beans property name patterns: setXXX(Object) and Object getXXX(), where XXX is the field name. This patten can co-exist with the present object-Dictionary approach.
All records are represented in memory as DBObjects (can extend Dictionary). To pass a pointer to an object, one just has to pass that DBObject. Such DBObjects can be registered into the Database (which will lead to an INSERT), and some of them can be given special names. Queries made in only one table return collections of DBObjects, and pointer and set fields will come as DBObjects.
DBObject methods for reading :
Fields are returned as:
Modifications (incl insertion) and destructions are performed inside transactions. Every thread is part of at most one transaction. A thread can join an already existing transaction. If a write operation is invoked, but no transaction is associated with the thread, a transaction consisting of only that operation will be performed.
Write operations will cause one INSERT or UPDATE for each affected object inside a transaction. INSERT is performed on an object with no pointer (and leads to setting that pointer), and UPDATE otherwise.
Object destruction: DBObject has a delete() method. All ptrOnes, setInt, setChar, setComplex will be deleted (in the same transaction).
A transaction can lock certain DBObjects for reading or writing. A transaction can be cancelled (abort, nothing is written to the database), or declared finished (commit, all changes are written). Also, the transaction can finish and re-start immediately, without releasing any locks.
At JDBC level, the Database will employ one Connection per transaction. Connections can of course be recycled.
The database has a cache, having the composite key [RecordInfo, ptr]. The cache contains data from the respective table (eventually in an array Object[] for each record).
The DBObject contains just a reference to the cache entry where its data stays. When the DBObject goes out of scope, the cache is notified that the respective entry has one less usage. A cache cleaner passes regularly and deletes cache entries which are not used.
To achieve this cache cleaning, based on the DBObjects going out of scope, one has to build a DBObject instance whenever a query result is returned, or pointer de-reference is made. That means that two DBObjects can represent the same thing, so changing a DBObject might affect another DBObject. To test if two DBObjects represent the same record in the database we can have a method such as
To avoid this, one has to use Java 2 (JDK 1.2) weak references. Using java 2 will also bring more standardization with the Collection classes. But for that, we need to upgrade to Java Web Server 2.0 (which might solve other problems we have now, such as page recompilation), currently Johnny and Minerva use Java Web Server 1.1.
Locking can be implemented at Java level or at database level.
For many of the classes (Database, Transaction) we could directly implement the ODMG interfaces. This will create a ODMG-like "metadata binding", which will give a standard value to metadata.
OODB discussion (Cristi and Stefan in Technico). This discussion probably came from the data-code relationship... In normal DBs, all objects of a type are accessible by a simple select. In OOP, objects are accessed thru references. This looks strange. Proposal to implement data inheritance with 2 extra columns (one table name and one pointer) in each table that points to a record containing the extension columns in separate extension tables. For example, a committee might request an extension to their archive, which in principle will behave like all other archives. This leads to thoughts about application inheritance, etc. In an OODB approach, viewing a table actually would view one single virtual set.
useful for development
|
BIGINT |
-5 |
|
constructor RecordFormatter(RecordInfo) |
initializes the formatter from a given metadata |
|
void writeInPage(Writer stream, Dictionary data) |
writes a view of the given data to the given stream |
|
void writeInForm(Writer stream, Dictionary data) |
writes an edit form of the given data to the given stream |
|
void writeInFormNew(Writer stream) |
writes an empty form (with default values) to the given stream |
|
Dictionary readFromQueryString(Dictionary query) |
reads the field from the http query |
adjustment methods (less urgent):
later:
|
void writeInFormSearch(Writer stream) |
writes a search form |
|
void writeInFormFormat(Writer stream) |
opens the formatter for editing to a non-tech user, as a edit form |
|
void writeInPageFormat(Writer stream) |
pens the formatter for editing to a non-tech user, as a page |
/best/student/$edit
/best/student/$view
/best/student/$new
/best/student/$list
are a tempting way to go. Then one just has to pass a pointer (for $view and $edit) to get results, which should be the defaul for $list views. A generic servlet interpreting this (reading the path to the metadata from its url path, reading the pointer and the form data from the query string, etc) can handle all cases needed, generating automatically some jhtml/jsp source for each case, which will then be adjusted by the webmaster.
Problems:
/organiser$/best/student/$view
then organiser could be a search in the auth and student tables
that would give a non-empty result if the respective is an
organiser
getFormatter().format(getData())
where getFormatter returns the formatter adjusted in the format() method, and format() returns the data identified by the pointer (for $edit and $view) or the bunch of data selected by $list.
/best/student/$view$format
This proposal was made for a fast implementation process and to give a well-known place for the java code.
Every JSP page that uses the Makumba taglibs will create a parallel Java class per operation invoked. The first page execution will call default implementations of the methods in the class and may announce the developer where to go to edit the code. The "pointer" to the code can stay in the generated HTML as a comment. It is intended that when the parallel code changes, the JSP is touched and re-compiled automatically. The JSP will call dedicated callbacks in the parallel class at different moments in the execution as illustrated below. References to the page and to the "brother classes" generated by the page will be available.
For example, a typical edit form will invoke the operations "writeInForm" and "edit" (since there are no response pages) So it will generate two parallel classes. Still, in such a case (presentation-response), one might have a single class dealing with the whole transaction, since the two are logically linked.
Still need to consider
- the webmaster-security of the solution
- the mechanisms by which the objects from the parallel class are
created and referenced
- our last dilemma in Coimbra: should the access rights and field
access rights be expressed in JSP-parallel or in DD-parallel (one
java class per Data description, i.e. Makumba file) java code?
- the clear definition of the methods. Below are some guidlines, but the final interface will come out per operation (view, view in form, edit, delete, insert) after some practice.
|
method signature |
semantics |
default implementation |
|
Object getInitialState() |
returns the initial state of an operation implementation. the state info will be passed to all methods of the template method |
returns null |
|
boolean isAuthorised(Pointer auth, Dictionary args, Object state) |
gets a pointer in an auth table detected by some http way |
if you are in the JSP root, return true |
|
Object getData(Pointer auth, Dictionary args, Object state) |
gets the data to view, view in form (pre-edit, pre-insert),
insert, or edit |
one http arg is a pointer, data is retrieved from the main table using it |
|
boolean beforeExcecute(Object state) |
Executes code before the operation (insert, delete, edit) is executed. Returns whether the operation should be executed or not. |
empty |
|
void afterExecute(Object state) |
similar to above |
empty |
|
boolean needTransaction(Object state) |
tells whether the operation and the rest should be wrapped in a transaction |
returns false |
|
boolean canView( String field) |
field access level. not clear if it will be here or not. |
returns true |
general/general_table1 (ex all countries, all languages)
/general_table2 ...
organization1/major_table1 (ex best/student)
/major_table2 ...
/application1/table1 ... (ex best/minerva/company)
/table2 ...
/application2/table1 ...
organization2/major_table1 ...
common_application/table1 ... (ex eventcalendar/event)
This is what the Stockholm meeting had in mind for BEST hierarchy. Object orientation and decoration were assumed
Auth Student best.Student decorates Student Group best.Lbg extends Group best.Committee extends Group best.vivaldi.Student decorates Student best.vivaldi.Season best.vivaldi.Activity best.vivaldi.Application best.minerva.Student decorates Student Company best.minerva.Company decorates Company best.minerva.CompanyRepresentative best.karamba.Document best.karamba.Message best.karamba.Repository best.karamba.OfficialArchive extends Repository best.karamba.CommitteeArchive extends Repository best.karamba.TigroBlabla extends CommitteeArchive