Mindoo Blog - Cutting edge technologies - About Java, Lotus Notes and iPhone

  • My latest wish list for the Domino Java API

    Karsten Lehmann  3 November 2016 11:10:49
    Last week I sent an email with enhancement ideas for the Domino Java API to IBM development. Most of the stuff listed here (except new @-functions) can be implemented by wrapping existing C API functions or simple copy&paste of already existing code.
    We already did this using our Domino JNA project, but I thought it would be a good idea to add this functionality to the standard API and provide full platform support.

    I don't really have unrealistic expectations whether this will ever be added to the product, given the latest speed new features got in, but I haven't given up hope yet.

    I am posting this here in the blog in case it is interesting for anyone.


    1. ViewNavigator View.createViewNavForSelection(int[] noteIds)
    Filter a view navigator to only return entries with specified note ids, e.g. received via previous lookups.

    Using the C API, this can be done by opening the view with NIFOpenCollection and populating the ID table "rethSelectedList":

    STATUS LNPUBLIC NIFOpenCollection(
    DBHANDLE hViewDB,
    DBHANDLE hDataDB,
    NOTEID ViewNoteID,
    WORD OpenFlags,
    HANDLE hUnreadList,
    HCOLLECTION far *rethCollection,
    NOTEHANDLE far *rethViewNote,
    UNID far *retViewUNID,
    HANDLE far *rethCollapsedList,
    HANDLE far *rethSelectedList);


    To navigate the view, NIFReadEntries would be called with NAVIGATE_NEXT_SELECTED / NAVIGATE_PREV_SELECTED flag.

    2. int[] View.getAllIdsByKey(Object key, boolean exact)
    Convenience function to only read the note ids of view entries matching the lookup key for better performance (less data in the summary buffer returned from NIFReadEntries, so more rows read with one C API call).

    Using the C API, this can be done with NIFReadEntries calls where only READ_MASK_NOTEID is used as return data mask.

    3. View.getViewEntryByKeyWithOptions(Object startKey, int searchOptions)

    This function is already available in the XPages runtime and would need to be copied over to the Domino Java API. The AbstractDominoViewPickerData class in the Extension Library uses it to locate view entries with a search key greater or equal "X":
    https://github.com/the-ntf/xspstarterkit/blob/master/extlib/com.ibm.xsp.extlib.domino/src/com/ibm/xsp/extlib/component/picker/data/AbstractDominoViewPickerData.java#L221

    ViewEntry ve=null;
    String startKey = options.getStartKey();
    if(startKey!=null) {
            int searchOptions = DominoUtils.FIND_GREATER_THAN|DominoUtils.FIND_EQUAL|DominoUtils.FIND_PARTIAL|DominoUtils.FIND_CASE_INSENSITIVE;
            ve = DominoUtils.getViewEntryByKeyWithOptions(view, startKey, searchOptions);
    } else {
            ve=nav.getCurrent();
    }


    So this gives us a way to do inequality searches for Strings and Numbers in views and decide if keys need to be case sensitive.



    These functions can be combined to implement powerful view lookups, e.g. filter view entries by multiple column values and sort the result dynamically.


    Here is an example:

    View myView = db.getView("MyView");
    myView.setAutoUpdate(false);

    //resort view by first sort column
    myView.resortView("firstsortcolumn", true);

    //get matches for first search key
    int[] noteIdsFirstKey = myView.getAllIdsByKey(firstSearchKey, false);

    //resort view by second sort column
    myView.resortView("secondsortcolumn", true);

    //get matches for second search key
    int[] noteIdsSecondKey = myView.getAllIdsByKey(secondSearchKey, false);

    //intersect both note ids lists: list1 AND list2 using some tool class
    int[] noteIdsFirstAndSecondKey = IDUtil.and(noteIdsFirstKey, noteIdsSecondKey);

    //resort view by third sort column
    myView.resortView("thirdsortcolumn", true);

    //get matches for third search key
    int noteIdsThirdKey = myView.getAllIdsByKey(thirdSearchKey, false);

    //merge intersection of first/second key matches with third: (list1 AND list2) OR list3
    int[] noteIdsFirstAndSecondKeyOrThirdKey = IDUtil.or(noteIdsFirstAndSecondKey, noteIdsThirdKey);

    //and now we produce the final sorting:
    myView.resortView("finalsortcolumn", true);

    //we would need this API to reduce the view to entries with specified note ids with max performance (let NIF skip irrelevant collection entries for us)
    ViewNavigator nav = myView.createViewNavForSelection(noteIdsFirstAndSecondKeyOrThirdKey);
    nav.setCacheGuidance(400, ViewNavigator.VN_CACHEGUIDANCE_READSELECTIVE);

    //skip some view entries to support paging
    nav.skip(start);

    //write requested page data (start/count) in JSON format:
    ViewEntry veCurrent = nav.getCurrent();
    while (veCurrent!=null && count>0) {
            if (veCurrent.isValid()) {
                    writeAsJson(veCurrent);
                    count--;
            }

            //read next view entry matching our selection list
            ViewEntry veNext = nav.getNext();
            veCurrent.recycle();
            veCurrent = veNext;
    }



    This view lookup technique is already being used by us at a big Domino customer in Germany using C calls (Domino JNA project), but platform support is currently limited to those supported by JNA and this should really be part of the standard API.



    Other useful APIs to add:

    4. @ModifiedInThisFile / @AddedToThisFile (posted in the Design Partner Forum in January 2014)
    Two new @-functions to sort view entries by the date their corresponding documents got added to/modified in the database. Required for synchronization of Domino data with external databases. Using @modified instead would not cover changes coming in via replication that happened before the cutoff date for the current data synchronization. Response to DP forum article in 2014 said the feature will be in 9.0.2, so it seems to be already implemented and would need to be added to a feature pack.

    5. NoteCollection.add(int[] noteIds)
    Fast bulk add function to add many note ids at once to a NoteCollection for better performance.

    6. int[] DocumentCollection.getNoteIDs()
    Fast method to get all note ids stored in a DocumentCollection as an int array. If DocumentCollection is sorted internally, the array should have the same sorting.
    Could be used in combination with "DocumentCollection Database.FTSearch()" to quickly get a note id list of documents matching a fulltext search.

    7. int[] Database.getDeletedDocuments(DateTime since, int noteClass)
    Method to get a list of note ids for documents deleted since the specified datetime. C API function NSFDbGetModifiedNoteTable already returns deleted notes ORed with RRV_DELETED flag. So this function would simply copy this list to an int array.
    Already available method Database.getModifiedDocuments(DateTime since, int noteClass) is no help because it filters out deleted documents.

    Use case is data synchronisation between Domino databases and with external systems.


    Comments

    1Michael BOURAK  03.11.2016 14:13:03  My latest wish list for the Domino Java API

    100% agree !