SIF Association

This blog is run by the staff of the SIF (Schools/Systems Interoperability Framework) Association for the use of it's members and the general educational community.

It is specifically designed for those who are interested in understanding what SIF is about, where the standard is going and how Schools, Districts and State Educational Authorities will benefit.

Monday, June 16, 2014

Coding the Code...

If you are a history and/or movie fan you may be familiar with the story of the Enigma machine.  Made popular in the movie U-571, the Enigma machine was invented at the end of WW1 and then utilized mostly by the Germans in WW2 to encipher and decipher wartime communications.  Eventually the Enigma machines functionality were undone by British cryptologist in a project codenamed “Ultra” speeding up the end of the war.

Encoding? Cryptology?  War?  What does all of this have to do with educational data management and usage?  A lot!  Pick up a newspaper today and you will see articles on data “breeches” from the National Security Agency, through retail marketplace providers all the way down to student information security concerns with the various data project alphabet soup we all are aware in the role we play supporting data usage.  Apparently there is a need for an “Ultra-proof Enigma” for the data we steward!
It is generally recognized that while data security and data privacy issues overlap to some degree but there are enough differences that data privacy concerns can effectively be treated separately.  Educational solutions at the school and district levels are increasingly deployed in the cloud, and that introduces a set of new concerns about who might be granted (or otherwise obtain) access to student health records, discipline actions or demographic identification information.
During the 2014 SIF Annual Meeting, at a session called “Who Owns the Data?”, a large group of school, state and vendor representatives determined a SIF Project Team was needed to focus on privacy issues surrounding access to and use of sensitive student related data.   This (international) Project Team will concentrate on end user (School, Local Authority, District and State) issues relating to student data privacy.  It will investigate the impact of educational cloud service providers and determine what set of “common privacy policies” they must commit to in order to meet, or where necessary exceed, FERPA and local data privacy mandates.
The work will address questions relating to:
  • Who “owns” the data?
  • Who “has” the data?
  • Who can access the data?
  • Who determines who can access the data?
  • Who can change the data?
  • Who defines “rules of ownership”?
  • Who enforces these rules?
  • Who verifies these rules are enforced?
  • Are there any special requirements when data is stored in 3rd party cloud?
The group will define a set of data privacy “effective practices” and document how those can be specified and enforced in solutions based upon openly developed technical standards usage in marketplace provider products and their end user customers.
Ask anyone, one of my favorite mantra is “You can’t have passion without participation!”  To participate in this critical work drop me a line at

Monday, April 21, 2014

A baker’s dozen reasons for adopting the SIF 3.0 infrastructure

The new SIF 3.0 Infrastructure is the powerful "on the wire" part of "CEDS on the wire" that leverages that Logical Data Model to generate conformant educational solutions delivering real functionality to students and staff.

Specifically it:

1.  Is independent of the payloads it conveys and the Data Model which defines them
a.    Identical Infrastructure carries payloads from all 3 locales and any future SIF Data Profiles
2.    Fully leverages industry standard technologies
a.    REST, XQuery, HMAC SHA256
b.    Standardizes both data query and data updates

3.     Can seamlessly support organizations implementing hybrid educational solutions containing both:
a.    Cloud based REST Web Services (via the Direct Environment)
b.    Enterprise Backbone Services (via the Brokered Environment)

4. Standardizes  enforcement of site-specific data security policies
a.    HTTPS and Transport Layer Security (TLS) for data encryption
b.    HMAC SHA256 for Client Authentication (shared secrets are never placed on the wire)
c.    Directly leverages Client Certificates where used

5. Standardizes  enforcement of site-specific data privacy policies
a.    Clear ACL-based Authorization restrictions defined by site administrators
b.    Data Privacy Profiles define restricted subset of existing objects (ex: Anonymous Student, with no identifying info like name, addresses, phone #s)
c.    Administrator-defined Zones enforce Data Privacy Profiles (ex: Zone X students are all Anonymous Students, which limits only those applications specifically assigned to that Zone)
d.    Additional Zones supporting other standardized privacy restrictions (ex: no Student health or disciplinary elements) may be simultaneously defined and used.

6. Standardizes support for massively scalable solutions
a.    Multiple objects can be contained in single message payload (ex: Attendance objects at the end of a reporting period)
b.    Event Pub/Sub support allows data synchronization without continuous polling overhead and message latency delays
c.    Message Queue enhancements cut asynchronous message traffic in half

7. Is modular
a.    The SIF “Environment” is constructed from a core set of mandatory service interfaces and a broader set of optional ones
b.    Many of the optional services can leverage existing middleware where available (ESBs, Service Registries, Queue Managers)

8.   Is robust
a.    XML payloads ensure message validation can be transparently enabled
b.    Standardized error handling and asynchronous error reporting
c.    Administrative level service interfaces allow automated detection of network connectivity problems and service failures

9.   Addresses real world use cases
a.    Service Paths allow common queries to be specified with single request (ex: return all students in school XYZ)
b.    XQuery scripts can be predefined to return customized objects reflecting the reporting requirements of the organization
c.    “Contexts (such as current or archived) for student objects directly support longitudinal data accessibility

10.  Standardizes application-to-application interoperability at multiple levels within the organization
a.    Tablet-based Dashboard to a school SIS (locally maintained or in the cloud) via a minimal Direct Environment
b.    Multiple applications within a small district communicating with a single SIS or Data Warehouse (locally maintained or in the cloud) via a full featured single Zone Direct Environment
c.    Multiple applications deployed in both multiple districts and at the state level interacting with multiple data sources (a hybrid solution with some services locally connected through supporting middleware and others deployed in the cloud) via a multi-zone Brokered Environment

11.   Is supported by a formal certification program to ensure application compliance and interoperability
a.    Separate levels of certification for infrastructure and educational application products
b.    Test Harness available to check that a software component under development is SIF compliant
c.    Certification Suite based upon that test harness validates whether claimed vendor functionality is actually present
d.    Certification Report for each certified product allows end users to select products with only the functionality they need

12.  Is developer friendly
a.    The “Read this First” document provides a documentation roadmap to the 3 volume SIF 3.0 Infrastructure specification
b.    The SIF REST Developers Sandbox offers immediate hands on experience with SIF-conformant message creation and analysis
c.    An on-going series of hosted Developer connect-a-thons brings implementers together to ensure their products interoperate.

13.  Is free of any IP encumbrances
a.    The SIF 3.0 infrastructure was created through the efforts of SIF Community vendors, integrators and  end users, utilizing an open standards development process
b.    There is no IP license to sign and no terms and conditions limit access to the technology
c.    Public feedback on the specification is continually solicited

Friday, April 4, 2014

SIF Implementation Specification 2.7: A Migration Release

A SIF 3 migration release is different than a SIF 2.x minor release in that it breaks backwards compatibility with earlier versions. SIF NA 2.7 incorporates important version 3 structural changes moving the data model towards compatibility with both the SIF 2.x and SIF 3 infrastructures. This document contains a summary of the new changes and describes how to leverage them to create an incremental and smooth migration path providing benefits at each step in the process while retaining compatibility with the SIF 2.x infrastructure. 
Using the SIF NA 2.7 data model to define the payloads carried by the SIF 3 infrastructure (and reaping the associated benefits) is straightforward, but considered experimental at this time, although sites interested in doing so are encouraged to provide feedback which will be used to prove and enable this migration approach.
Universal RefId Usage
One of the most developer requested features over the years has been to add a unique identifier field to the format of every object type. However changing the existing object structure to incorporate a new root level attribute has been a breaking change impossible to introduce under SIF’s policies defining what is possible in a minor release. SIF 3’s strong REST underpinnings have completely changed this conversation, as without an identified unique key for each item in a collection the REST design pattern cannot function. In order to move SIF 2.7 over the SIF 3 infrastructure, these long sought after SIF object identifiers (RefIds) have been universally added to every supported object type.
This is a well understood and bounded breaking change. The benefits of improved referential integrity are well known and the expected break is limited to a single situation which has a known solution. Applications already deal with new unexpected data fields added during the normal minor release process. The typical agent SHOULD consume only the data it understands from any incoming object. While this can be achieved several ways depending on the tools employed, logically it makes the most sense to design message reception logic which detects and processes only a list of (simple) XPaths corresponding to XML elements of interest for every supported object type. The result of this approach is an agent which only consumes only the subset of elements it needs in an arriving object, regardless of any other elements which are contained there as well.
This sort of pre-SIF 2.7agent will continue to function, unchanged even if a RefId is added to an object by an agent conformant with SIF 2.7 or greater. This however solves only one of several interoperability use cases.
A second problem arises when a SIF 2.7 or greater agent receives one of the impacted objects from a pre SIF 2.7 agent. Now a mandatory field (the RefId) is missing and MUST be generated by the message recipient. Where the recipient is the SIF 2.7 object provider (the majority of cases) this is expected to work quite well; especially as it conforms to the SIF 3.0 infrastructure requirement that only the object provider may assign the RefId.  The final case is where the SIF 2.7 agent recipient is not the object provider and receives a change request (create, modify or delete) which does not have a RefId. In that situation the agent may query for the object utilizing other (ideally unique) keys, storing the RefId returned if the object provider is SIF 2.7 or above. Either way, the optimum strategy for a SIF 2.7 agent receiving impacted SIF objects SHOULD be to first match by RefId and then by any xs:unique keys before concluding the received is a different object.
Another clear benefit of installing SIF 2.7 components is their support for new and updated object types that reflect their SIF 3 counterparts, providing new functionality and easing future transitional steps. The SIF 2.7 release includes an updated version of the SIF 3 Assessment objects and the ground breaking set of Identity Management objects.  The North American Technical Board may choose to create an additional SIF 2.x migration release based upon the SIF 3 infrastructure in the future, in order to provide existing SIF 2.x adopters with the enhanced functionality contained in the new infrastructure and in the on-going data model work.  The SIF 2.7 release then represents the first major step on the path to SIF 3.0 functionality. 
The SIF Association has released a SIF 2.7 Certification Suite that certifies application conformance with the combination of the SIF NA 2.7 data model release and the SIF 2.x infrastructure. Specific areas of the earlier SIF 2.x infrastructure that were incompatible with the SIF 3 infrastructure have been deprecated and will not be tested. These include Zone Services, the SOAP Transport, and Bundled Events.
The SIF Data Model Implementation Specification (North America) 2.7M release is available from the SIF website.

Tuesday, November 12, 2013

SIF 3.0 Infrastructure (Very) High Level Overview

The following is taken from the Read This First (RTF) document, which forms the introduction to the SIF 3.0 Infrastructure release collateral.  For those interested in exploring the new SIF release in further detail, the RTF is probably the best place to begin.  It is located at:


Q. What exactly IS the SIF 3.0 Infrastructure?

SIF 3.0 is the latest release of an open standard infrastructure which began 15 years ago as the product neutral interface of an existing commercial message broker. This release brings SIF into the modern era by leveraging a REST based approach to data exchange. The key contribution the SIF 3.0 release makes is to define, coordinate and standardize the ways in which a RESTful educational service can be accessed securely, robustly, and in real time by multiple RESTful clients.

SIF 3.0 infrastructure is the first infrastructure release to be completely separate from the data model defining the payloads it carries, which means it can be used to support many different data models in many different locales. For example while the SIF US data model is based on CEDS, this is not explicitly reflected in the SIF 3.0 Infrastructure documentation.

Q. Why was a major release of the infrastructure needed at this time?

The SIF 2.x infrastructure was initially architected more than a decade ago, and the SIF 3.0 infrastructure introduces a wide range of needed new functionality. Of particular note are three ground-breaking design advances which satisfy long standing requests from SIF 2.x developers and implementers. They are summarized below. 

1. Make the SIF Infrastructure independent of the SIF Data Model

All current data model dependencies of the earlier release have been removed.  As a result, the SIF 3.0 infrastructure can carry SIF object data from any locale (US, UK, AU), or for that matter data in conformance with other major data standards, without change.


2. Offer alternatives and extensions to the required monolithic middleware component (ZIS)

This issue has been addressed on two fronts. 

First, the ZIS-provided message broker functionality has been broken up into a set of multiple, separately implementable Infrastructure Services.  These were designed to individually map to common industry technologies such as “Enterprise Service Bus (ESB)”, “Queue Manager” and “Service Registry”.  The SIF 3.0 Brokered Environment middleware can still be implemented by a single, monolithic component, but it no longer has to be. 

Second, the new SIF 3.0 architecture makes it possible to construct and deploy SIF-compliant solutions in a Direct Environment without utilizing any middleware at all!  This new SIF infrastructure alternative offers clients a standardized subset of the functionality available from the Brokered Environment, which means that these clients can be deployed into middleware-centric solutions with no recoding or reintegration efforts required.


3. Support industry standard transport technologies

The SIF 3.0 infrastructure documentation describes the service framework and associated core services and utilities in a platform neutral manner.  As a result, it can be mapped to any modern transport running over HTTP/S.

The defined platform mapping of the SIF 3.0 infrastructure is REST.  The SIF 3.0 infrastructure includes paged reads, synchronous IO and support for the primary REST resource design patterns.  XQuery scripts and dynamic Query URL parameters are both supported in SIF 3.0 replacing the earlier SIF-specific (and less powerful) Query and Extended Query functionality.

These and other changes allow SIF 3.0 solutions to be deployed in the Data Center of an educational organization using the identical technologies that are already present and known to IT personnel.  It also makes it easier for vendors to staff SIF-related projects as both the REST infrastructure technology and to a lesser extent.

SIF 3.0.  Everything you want in a SIF Infrastructure.

Tuesday, September 10, 2013

It's here...!

The SIF Association is proud to announce the release of the SIF Implementation Specification 3.0 to the educational marketplace.  This latest release, developed openly by the dedicated community volunteers, is a data standards ‘game changer’ - offering ground-breaking transport design advances leveraging REST technology, and providing a ground up redesigned data model which can support multiple data initiatives!

This latest release - SIF Implementation Specification (North America) 3.0 - is made up of a globally utilized reference infrastructure and North America data model focusing on supporting the Common Education Data Standards initiative (CEDS). The new 3.0 infrastructure allows the transport of various data models including those from the other global SIF communities as well as data from the numerous “alphabet soup” data initiatives that are populating the education landscape.  In essence – education now can utilize “one wire with one plug” – not the never-ending proprietary API’s and “one off” connections!

To read the full press release, click here.

To find out more information, including 'What's New at SIF?', RFP language, and much more, click here.
To review the SIF Implementation Specification (North America) 3.0, click here.
To review the SIF Infrastructure Specification (Global) 3.0, click here.


Thursday, May 2, 2013

There's "common" and then there's "common"...

You may or may not have been keeping up to speed on pushback in regards to the developing Common Core State Standards (CCSS). These high level academic content standards are designed to provide a framework of “…what students are expected to learn, so teachers and parents know what they need to do to help them”. I have intimate knowledge of all of the political pitfalls of developing learning standards during my tenure in Ohio – it can be frustrating and alienating stuff!
So why does a technical standards organizational lead need to write on a topic that does not directly impact the development of interoperable blueprints for information transport?  The rumblings against the “Common Core” have now bled over to generate misperceptions around the Common Education Data Standards (CEDS). Like the CCSS, the CEDS project has engaged a large number of early childhood, K12 and higher education, software companies and government agencies. Unlike the CCSS, CEDS is developing a dictionary of common data definitions already used by schools/states and not common learning outcomes. CEDS is a common vocabulary blueprint that people can use in building their local and state data systems – it collects nothing and reports nothing. It works better to have one agreed to and collaboratively developed dictionary – not 50!
So why this topic in a SIF blog?  CEDS does not standardize how the various data sources located within districts or states transfer their data. The SIF Implementation Specification (US) 3.0 defines a dynamic application-to-application data transfer framework, which is secure, robust, and fully CEDS-compatible – plus more! The SIF data model continues to address the needs not outlined in CEDS but needed locally like transportation, food service, library, HR, etc. It has been doing that successfully for 15 years! While enabling various needs in the states, the SIF community must also allow for local data model needs for our UK and AU communities where CEDS is not a driving force. 
Version 3.0 contains “buckets” to carry academic learning standards (i.e. CCSS, individual LEA or SEA standards, etc.) allowing for local control of what students know and should be able to do and yet leverages common definitions of what the data is in use at all levels. Sometimes “common” is not that common...

Larry Fruth
Executive Director/CEO
SIF Association

Wednesday, April 17, 2013

SIF 3.0 first look: Embracing new standards and getting it right!

The excitement is palpable, the will is good, and the details are finally coming into the light. SIF 3.0 is no longer just vapor; large chunks are now publicly accessible in release candidate form.

How large? You be the judge of that.

  • Base Architecture (updated, some open issues)
  • Infrastructure Services
  • Initial Global Data Model
  • Core U.S. CEDS Aligned Data Model
  • SIF-RS (on the wire SIF 3.0 REST Mapping)

As you consider the above, please keep a careful eye on the following:
  • Internationalization
    • One infrastructure multiple data models
    • XML Namespaces pervasive and complete
    • Conducive to JSON environments
  • For the US Community - 100% CEDS
    • Fully aligned
    • Efficient on the wire
    • Eventually complete (early childhood through workforce)
  • Simple
    • Works with various libraries and frameworks
    • Favors the Data Consumer, Data Provider, Direct Zone Provider then Middleware
    • Modern (token based) client authentication system
  • Scalable
    • Reduces chattiness
    • Overcomes latency
    • Prioritizes traffic
    • Empowers flow control

So if you are an early adopter, working on a proof of concept, or just want to help refine by trying, start here:

Improvements are welcome!