SIF Association

This blog is run by the staff of the SIF (Schools/Systems Interoperability Framework) Association for the use of it's members and the general educational community.

It is specifically designed for those who are interested in understanding what SIF is about, where the standard is going and how Schools, Districts and State Educational Authorities will benefit.

Monday, April 21, 2014

A baker’s dozen reasons for adopting the SIF 3.0 infrastructure

The new SIF 3.0 Infrastructure is the powerful "on the wire" part of "CEDS on the wire" that leverages that Logical Data Model to generate conformant educational solutions delivering real functionality to students and staff.

Specifically it:

1.  Is independent of the payloads it conveys and the Data Model which defines them
a.    Identical Infrastructure carries payloads from all 3 locales and any future SIF Data Profiles
2.    Fully leverages industry standard technologies
a.    REST, XQuery, HMAC SHA256
b.    Standardizes both data query and data updates

3.     Can seamlessly support organizations implementing hybrid educational solutions containing both:
a.    Cloud based REST Web Services (via the Direct Environment)
b.    Enterprise Backbone Services (via the Brokered Environment)

4. Standardizes  enforcement of site-specific data security policies
a.    HTTPS and Transport Layer Security (TLS) for data encryption
b.    HMAC SHA256 for Client Authentication (shared secrets are never placed on the wire)
c.    Directly leverages Client Certificates where used

5. Standardizes  enforcement of site-specific data privacy policies
a.    Clear ACL-based Authorization restrictions defined by site administrators
b.    Data Privacy Profiles define restricted subset of existing objects (ex: Anonymous Student, with no identifying info like name, addresses, phone #s)
c.    Administrator-defined Zones enforce Data Privacy Profiles (ex: Zone X students are all Anonymous Students, which limits only those applications specifically assigned to that Zone)
d.    Additional Zones supporting other standardized privacy restrictions (ex: no Student health or disciplinary elements) may be simultaneously defined and used.

6. Standardizes support for massively scalable solutions
a.    Multiple objects can be contained in single message payload (ex: Attendance objects at the end of a reporting period)
b.    Event Pub/Sub support allows data synchronization without continuous polling overhead and message latency delays
c.    Message Queue enhancements cut asynchronous message traffic in half

7. Is modular
a.    The SIF “Environment” is constructed from a core set of mandatory service interfaces and a broader set of optional ones
b.    Many of the optional services can leverage existing middleware where available (ESBs, Service Registries, Queue Managers)

8.   Is robust
a.    XML payloads ensure message validation can be transparently enabled
b.    Standardized error handling and asynchronous error reporting
c.    Administrative level service interfaces allow automated detection of network connectivity problems and service failures

9.   Addresses real world use cases
a.    Service Paths allow common queries to be specified with single request (ex: return all students in school XYZ)
b.    XQuery scripts can be predefined to return customized objects reflecting the reporting requirements of the organization
c.    “Contexts (such as current or archived) for student objects directly support longitudinal data accessibility

10.  Standardizes application-to-application interoperability at multiple levels within the organization
a.    Tablet-based Dashboard to a school SIS (locally maintained or in the cloud) via a minimal Direct Environment
b.    Multiple applications within a small district communicating with a single SIS or Data Warehouse (locally maintained or in the cloud) via a full featured single Zone Direct Environment
c.    Multiple applications deployed in both multiple districts and at the state level interacting with multiple data sources (a hybrid solution with some services locally connected through supporting middleware and others deployed in the cloud) via a multi-zone Brokered Environment

11.   Is supported by a formal certification program to ensure application compliance and interoperability
a.    Separate levels of certification for infrastructure and educational application products
b.    Test Harness available to check that a software component under development is SIF compliant
c.    Certification Suite based upon that test harness validates whether claimed vendor functionality is actually present
d.    Certification Report for each certified product allows end users to select products with only the functionality they need

12.  Is developer friendly
a.    The “Read this First” document provides a documentation roadmap to the 3 volume SIF 3.0 Infrastructure specification
b.    The SIF REST Developers Sandbox offers immediate hands on experience with SIF-conformant message creation and analysis
c.    An on-going series of hosted Developer connect-a-thons brings implementers together to ensure their products interoperate.

13.  Is free of any IP encumbrances
a.    The SIF 3.0 infrastructure was created through the efforts of SIF Community vendors, integrators and  end users, utilizing an open standards development process
b.    There is no IP license to sign and no terms and conditions limit access to the technology
c.    Public feedback on the specification is continually solicited

Friday, April 4, 2014

SIF Implementation Specification 2.7: A Migration Release

A SIF 3 migration release is different than a SIF 2.x minor release in that it breaks backwards compatibility with earlier versions. SIF NA 2.7 incorporates important version 3 structural changes moving the data model towards compatibility with both the SIF 2.x and SIF 3 infrastructures. This document contains a summary of the new changes and describes how to leverage them to create an incremental and smooth migration path providing benefits at each step in the process while retaining compatibility with the SIF 2.x infrastructure. 
Using the SIF NA 2.7 data model to define the payloads carried by the SIF 3 infrastructure (and reaping the associated benefits) is straightforward, but considered experimental at this time, although sites interested in doing so are encouraged to provide feedback which will be used to prove and enable this migration approach.
Universal RefId Usage
One of the most developer requested features over the years has been to add a unique identifier field to the format of every object type. However changing the existing object structure to incorporate a new root level attribute has been a breaking change impossible to introduce under SIF’s policies defining what is possible in a minor release. SIF 3’s strong REST underpinnings have completely changed this conversation, as without an identified unique key for each item in a collection the REST design pattern cannot function. In order to move SIF 2.7 over the SIF 3 infrastructure, these long sought after SIF object identifiers (RefIds) have been universally added to every supported object type.
This is a well understood and bounded breaking change. The benefits of improved referential integrity are well known and the expected break is limited to a single situation which has a known solution. Applications already deal with new unexpected data fields added during the normal minor release process. The typical agent SHOULD consume only the data it understands from any incoming object. While this can be achieved several ways depending on the tools employed, logically it makes the most sense to design message reception logic which detects and processes only a list of (simple) XPaths corresponding to XML elements of interest for every supported object type. The result of this approach is an agent which only consumes only the subset of elements it needs in an arriving object, regardless of any other elements which are contained there as well.
This sort of pre-SIF 2.7agent will continue to function, unchanged even if a RefId is added to an object by an agent conformant with SIF 2.7 or greater. This however solves only one of several interoperability use cases.
A second problem arises when a SIF 2.7 or greater agent receives one of the impacted objects from a pre SIF 2.7 agent. Now a mandatory field (the RefId) is missing and MUST be generated by the message recipient. Where the recipient is the SIF 2.7 object provider (the majority of cases) this is expected to work quite well; especially as it conforms to the SIF 3.0 infrastructure requirement that only the object provider may assign the RefId.  The final case is where the SIF 2.7 agent recipient is not the object provider and receives a change request (create, modify or delete) which does not have a RefId. In that situation the agent may query for the object utilizing other (ideally unique) keys, storing the RefId returned if the object provider is SIF 2.7 or above. Either way, the optimum strategy for a SIF 2.7 agent receiving impacted SIF objects SHOULD be to first match by RefId and then by any xs:unique keys before concluding the received is a different object.
Another clear benefit of installing SIF 2.7 components is their support for new and updated object types that reflect their SIF 3 counterparts, providing new functionality and easing future transitional steps. The SIF 2.7 release includes an updated version of the SIF 3 Assessment objects and the ground breaking set of Identity Management objects.  The North American Technical Board may choose to create an additional SIF 2.x migration release based upon the SIF 3 infrastructure in the future, in order to provide existing SIF 2.x adopters with the enhanced functionality contained in the new infrastructure and in the on-going data model work.  The SIF 2.7 release then represents the first major step on the path to SIF 3.0 functionality. 
The SIF Association has released a SIF 2.7 Certification Suite that certifies application conformance with the combination of the SIF NA 2.7 data model release and the SIF 2.x infrastructure. Specific areas of the earlier SIF 2.x infrastructure that were incompatible with the SIF 3 infrastructure have been deprecated and will not be tested. These include Zone Services, the SOAP Transport, and Bundled Events.
The SIF Data Model Implementation Specification (North America) 2.7M release is available from the SIF website.