SIF Association

This blog is run by the staff of the SIF (Schools/Systems Interoperability Framework) Association for the use of it's members and the general educational community.

It is specifically designed for those who are interested in understanding what SIF is about, where the standard is going and how Schools, Districts and State Educational Authorities will benefit.



Wednesday, November 28, 2012

Lions, Tigers and Data Models... Oh My!

CEDS on the Wire and SIF Data Models


With the NCES’s Common Education Data Standard (CEDS) v3 release upcoming after the first of the year, it is important now for the SIF community to elaborate on the "CEDS on the Wire" motto for the SIF US 3.0 Data Model.  Last January the US Management Board leadership, lead by LEA, SEA and marketplace providers, set the strategic direction for the community to use our model driven approach to include CEDS as the core for the maturing SIF Data Model in the United States.
SIF Implementation Strategy for CEDS

The CEDS Model is an integral part of the SIF Implementation Specification Data Models.  More specifically, CEDS is a key component in the SIF Logical Data Model.  This logical model forms the basis for all implementable SIF data models.
Beginning with its 3.0 Specification, SIF uses a series of intermediary data models in order to produce implementable data models.  These data models use a “stack” approach to building an implementable model. And while CEDS is not the only important building block in the SIF Data Model Stack, it is 100% represented in the SIF Implementation Specification Data Model.
The stack represents data models along the conceptual -> logical -> physical data model continuum.  In this paradigm, the SIF Entity Model and the SIF Payload Model together make up the SIF Implementation Specification. 
 


Some important points about the SIF Data Model Stack:
  • The SIF approach to data models includes the continuum from conceptual/information models to physical models.
  • Conceptual/Logical models (a.k.a., information models) are bound (or serialized) to a particular technology such as XML for implementation. We are representing the logical model in UML as an intermediate step toward XML that is free from the technical and implementation concerns of XML.
  • Physical models are implementation models because they take use cases, serialization method (e.g. XML) and locale-specific usage into account.
  • Only physical data models are implementable, where conceptual and logical data models are not.

How are the SIF Data Models Created?

The Model Driven Approach builds successive data models on the solid foundations of conceptual and logical models.  These foundational models are necessary and convenient because they allow us to talk about education data without the interference of database technology or implementation concerns.  These concerns are addressed later during physical data model discussions.
SIF has a strong tradition of a use-case-driven approach to development.  This approach is also incorporated into the SIF development process to insure that real-world needs are addressed in the final data model. This is accomplished using the model driven architecture (MDA) approach illustrated below.



CEDS on the Wire

The SIF 3.0 Data Model Implementation Specification will provide a flexible and complete data model with which to implement CEDS.  In addition, the SIF 3.0 Infrastructure Specification will provide many options and components for the transport of CEDS data elements.

The SIF Association remains a strong supporter and participant in CEDS.  The Association participates in the development of CEDS and is committed to providing a comprehensive data model that incorporates all of the CEDS data elements. Stay tuned for the release of the SIF Version 3.0 Data Model Implementation Specification soon after the release of CEDS v3.

Saturday, October 13, 2012

SIF v3.0 Infrastructure Revisited



This is admittedly a rather long post, but for those interested in understanding the three groundbreaking SIF infrastructure design decisions made at the SIF Fall Developer Camp in Dallas last month, it should definitely be worth the read.

 Each decision has had a huge impact on the capabilities of the SIF v3.0 release, and each is summarized below.

1.    REST is now a “co-equal” transport to SOAP

REST joins SOAP as the second technology that can be used to provide the foundation for exchanging SIF-compliant data.  The specific changes needed to make this happen are already underway, and include:

  •  The SIF v3.0 infrastructure Service Architecture has been aligned with the Get/Post/Put/Delete standard REST operations to make them more easily map-able to a set of standard REST resources.
  • A new “immediate return” alternative for Request / Response exchanges is expected to be heavily utilized by REST applications.
  •  A separate SIF v3.0 infrastructure volume will contain the agreed on set of REST Developer Guidelines (including URL allocation) that will guarantee REST applications can smoothly interoperate in a SIF v3.0 Zone.
  • An additional infrastructure volume targeted at SIF v3.0 Middleware providers will define the complete bridging instructions between REST and SOAP messages (similar to the SIF v2.x volume which bridged SOAP and the classic SIF transport).  This ensures the choice of transport will not fragment SIF application interoperability.
 
2.    There is a new Zone alternative in which the Client Application connects directly with the Service Provider

In addition to the Brokered (3-party) SIF Zone architecture which had previously defined all SIF v2.x deployments, there is now a new Direct (2-party) alternative in which a client and service can exchange SIF-conformant data directly without the need to interpose message broker middleware such as the ZIS.  Essentially a “very thin” Zone Provider wrapper is placed around an Object Service, to allow the “Direct Zone” to be provided by an SIS or LMS system.

This wrapper (the “Common API”) is a subset of the “Extended API” wrapper supported by all Brokered Zones, which provides access to a broader set of middleware services. As a result, every SIF v3.0 client application which was written to function in a Direct Zone will also function in a SIF v3.0 Brokered Zone.

Here are two use cases for a Direct Zone that are particularly illuminating:
  •  When an application wants to make its SIF-compliant data accessible to a user running a simple RESTful client application on a mobile device like an iPad (much as it might already be available to that user via a browser).
  • When an application only needs data from one other application.  An example is a Student Contact application needing to access Student Name and Addresses and Phone Numbers from an SIS system. If the SIS system provides a Direct Zone, it enables the Student Contact system to access its data directly without the need to install middleware.
All applications (client or service) in the above examples can be SIF-certified, and there is every expectation that each will interoperate “out of the box” with other partners supporting the Direct Zone APIs.
 
The differences between Direct and Brokered SIF v3.0 Zones are further detailed in a table below.

3.    Migration to SIF v3.0 will be a far cleaner 2-step process than earlier envisioned.  

While it was always the plan to eliminate any Data Model dependencies in the SIF v3.0 infrastructure, what is new (and which seems obvious in retrospect) is the realization that such a “data payload” independent infrastructure can also be used to support exchanges of SIF v2.x data.

As a result, migration to SIF v3.0 will now be a far cleaner 2-step process than earlier envisioned.  A SIF Zone administrator begins by first upgrading the existing Zones in her organization to the new infrastructure (benefiting from the new alternative choices described above).  Only at that point does the need arise to address legacy application migration from the SIF v2.x Data Model to the new CEDS-based (in the US) SIF v3.0 Data Model.


                           <And now ... for some details>
As promised, here is the table highlighting the detailed differences between the two SIF Zone types

Alternative / Functionality

SIF Direct Zone
SIF Brokered Zone
Targeted SIF Data Model Version Support

SIF v2.x or v3.0
SIF v3.0
Transport Support & Certification
REST only in support of the SIF v2.x Data Model.

REST or SOAP in support of the SIF v3.0 Data Model.
REST and SOAP in any given SIF v3.0 Brokered Zone

Summary
SIF Zone without the middleware
Fully functioned SIF v3.0 Zone

Middleware required?

No.  All connections are 2-Party
Yes.  All data exchanges are mediated by a 3rd Party Broker

Client to Service connections supported
One Client to one or more Services from a single Provider

Many Clients to many Services from many Providers
Expandability 
The Application supplying the Direct Zone may be written to simultaneously provide Direct Zones to multiple clients.


New concept of “subzones” effectively allows single middleware module to support (and a single Zone administrator to manage) hierarchical groupings of communicating clients and services

Client to Service(s) session supported

Yes. Secure session support provided by single Application Service
Yes.  Secure session support provided by the Broker

Client flexibility

Medium.

The Registered Client in a Direct SIF v2.x Zone must use the REST transport.  .

The Registered Client in a Direct SIF v3.0 Zone must use a transport which matches that of the Application offering the Zone (i.e. either SOAP or REST).

JSON to XML converters may be used by REST clients before sending or after reception, but only XML Payloads will be transmitted “over the wire”.

All Request / Response exchanges are immediate (Response is returned in the HTTP Response rather than “ACK of Request”)).

High.  Registered Clients in any given Brokered Zone can be:

REST and / or SOAP.  The Broker does all transport bridging

As with Direct Zones, only XML-formatted Payloads are exchanged “over the wire”. 

Request / Response exchanges are either immediate or delayed depending upon client and service preference at registration time.  The Broker does any necessary bridging.

Client Functionality

Medium 

The Client can access the Direct Zone to query and update only those data objects provided by the Application implementing the Zone. The Client cannot declare itself as a Service.

Even if the application providing the Direct Zone supports multiple Clients, each client is effectively placed in its own Zone.

Cross-client communication is only possible through Events ... if one client updates a supported Service object, every client in every other Direct Zone supported by the Service Application will see the corresponding Change Event.

Very high.

As with the Direct Zone, Clients can see the Events resulting from each other’s Service invocations, 

Clients can also dynamically provision themselves as services and can then be queried and updated by all registered clients in the Brokered Zone with the proper Authorization rights.

Clients can discover other clients in the Zone.

They can open multiple queues to prioritize certain message types and increase Zone scalability.

Multiple Subzones (with multiple default providers) are supported, as are additional Utility Services like Extended XQuery and Alert Log.

A “Zone Management” Client can discover and verify the access settings of all other Clients and Services, monitor their performance, and detect their errors.

Reduces barriers for new vendors

Considerably.

A SIF v2.x application can put a thin interface in front of itself by slightly extending the existing SIF Agent  APIs, whereupon it can directly service requests and post events to other SIF compliant clients in a Direct Zone without needing a middleware component.



Somewhat.

Infrastructure APIs are repackaged to correspond to middleware architectural components.

This promotes possible construction of the SIF v3.0 Message Broker functionality from individual components in a middleware product suite as well as from an upgraded SIF v2.x ZIS.

Relation to other alternatives
The Direct Zone supports a clearly defined subset of the Brokered Zone API. 

Since it is independent of the Data Model defining the payloads it transports, it can be applied to conveying SIF v2.x data and well as SIF v3.0 data.
The Brokered Zone “bridges” between the SOAP and REST transports, and extends but does not replace the functionality provided to Direct Zone clients.

As a result, an application (REST or SOAP) which is a client of any Direct Zone should seamlessly interoperate with all other applications in any Brokered Zone supporting the same SIF version of the data payloads.

Manageability

Low to medium.

Basically management tools are whatever the application developers provide, and traditionally this is less extensive than management functionality provided by a ZIS, ESB or Service Registry.
Very High. The SIF v3.0 Brokered Zone extends the remote management functionality of earlier releases (including both preventative maintenance and security) to allow effective central administration of multiple Zones by providing:

* Retrieval of Alert Log information through straightforward Queries (ex: it is now possible to find out what errors were logged by or against application X)

* Query capability of the Queue Manager statistics to determine the number and total size of pending messages, and the time of last message delivery for each client and service.

This should be extremely useful in predicting and then isolating application-specific problems within the Zone.







Friday, September 14, 2012

SIF v3.0 Infrastructure High Level Feature Set


The SIF v3.0 infrastructure (the “wire” part of “CEDS on the Wire”) provides some powerful and long awaited enhancements to the “SIF classic” HTTP/S infrastructure, which today provides a secure and reliable message exchange mechanism for applications interoperating in a SIF v2.x Zone.

These infrastructure enhancements all conform to the set of infrastructure release “Commandments” discussed in an earlier posting, and they are highlighted below.

1 Provide total independence from the SIF Data Model Schema

The existing SIF v2.x SOAP/WSDL transport provides the basis of the SIF v3.0 infrastructure, eliminating the data model dependencies present in the “SIF classic” infrastructure. This allows the new infrastructure to carry SIF object data from any locale without change, or for that matter, to carry data payloads defined by any standard that is independent of the infrastructure that transports it.

2 Increase Scalability

Several existing SIF v2.x performance limitations are being addressed and mitigated. This includes combining the 2-step handshake around every message received by a Pull Mode Agent into a single message exchange between Client and Service, which can double Pull Mode Agent throughput in periods of high message traffic. 

3 Support Direct REST Client / Server connections

This new functionality, which among other things extends the range of SIF conformant data exchanges to include applications running on mobile devices, was described in a previous posting.

4 Align Infrastructure Services APIs with middleware components

The service interfaces to SIF v3.0 infrastructure functionality have been re-structured to more closely match the individual services defined by middleware architecture (such as an ESB, Service Registry and Queue Manager).  As a result it will be easier for middleware architects to understand the functionality required to support a SIF Zone, and more straightforward to construct SIF infrastructure solutions based upon middleware product suites.

5 Define process “behavior” as well as data exchange formats

SIF v3.0 Functional Services generalize the older Zone Services and provide a way to define and encapsulate the behavior of educational processes so that the underlying message exchange sequences are made invisible to clients.  Some examples of currently targeted Functional Services include Enroll Student, Transfer Student and Score Test Results.

6 Support XQuery functionality

The current SIF-specific Extended Query message defined in SIF v2.x will be replaced in SIF v3.0 with the industry standard and more powerful XQuery functionality.

7 Standardize Zone Diagnostic and Preventative Maintenance “hooks”

SIF v3.0 defines additional “Zone Management” functionality designed to help central administrators dynamically monitor and proactively improve the overall health of the zones they manage.  This includes the ability to actively track Zone bottlenecks, load imbalances and application failures.



Several of these infrastructure advances will be described in more detail in the coming weeks, along with the SIF v3.0 migration strategy which outlines how existing vendors and end users can transition their SIF-compliant products and zones to the new release.

The complete 3.0 infrastructure specification itself is expected to be published in late January 2013.

 

Wednesday, August 22, 2012

Transport 'contenders': REST and SOAP/WSDL


There are currently two leading transport “contenders” in the technology space today: REST and SOAP/WSDL.  The trick for SIF has been how to leverage both effectively without fragmenting the SIF community into two isolated sets of SIF clients and services which did not interoperate with each other.

The solution chosen for SIF 3.0 consists of two clear and distinct strategies.

-       Use SOAP WSDL conveying XML message payloads where it is strongest ... in the Zone, where all data exchanges are brokered and asynchronous, the level of security that must be provided is high, and the set of SOAP tool support for the architecture is extensive

-       Leverage REST (whether conveying XML or JSON message payloads) to extend the ability to exchange SIF-compliant data to cases where the client and service are directly connected (i.e. outside the Zone).

While “SOAP/WSDL in the Zone” is a natural progression of the same functionality already supported from SIF 2.5 on, the embracing of REST technology is new to SIF.  It is being issued as a series of developer guidelines, and it utilizes REST to address use cases such as the following:

A Student Information System (SIS), or Management Information System (MIS), is a member of a SIF zone (i.e. it already has a SIF Agent which implements the SIF brokered connection to other applications).  It has an internal browser interface which end users utilize to access and enter student data.

Now add the following requirements:

-       The developers want to extend their user interface to allow teachers to securely access student data from mobile devices like an iPad rather than just a browser

-       There are small applications that only want access to the SIS/MIS data (ex: Student Contact), and have no other reason to provide themselves with SIF Agents 

-       The SIS/MIS can now add the logic to provide the SIF “RESTful Service” interface to support synchronous SIF-compliant data interchanges, allowing the development of a simple REST client conforming to the published SIF guidelines to satisfy both of the above requirements.

Many SIS/MIS have direct REST client support today ... this standardizes their support of front ends in mobile devices or communications with separate small applications.  When written to the published guidelines, these REST clients can now access any SIF-compliant SIS/MIS supporting this new interface.  The new Direct REST connection enhances rather than replaces the brokered SOAP interface the SIS/MIS is using.

The best of both worlds in other words…


Monday, July 23, 2012

SIF v3.0 Infrastructure "Commandments"

Work is currently well underway on defining the new SIF v3.0 infrastructure ... the "wire" part of "CEDS on the Wire".  It is being guided by the following set of informal development “Commandments”, which give a flavor of what that wire will look like once it it completed.

The SIF v3.0 infrastructure:
  •  Shall leverage functionality now provided by modern middleware technologies 
  •  Shall provide a clear migration path for current suppliers and end users of SIF infrastructure technology
  • Shall lower the barrier to entry for new vendors, thereby helping to increase the number of new SIF infrastructure technology providers.
  •  Shall not drop any existing functionality (whether optional or mandatory) in the current release which has been utilized by significant numbers of SIF end users, without supplying a viable alternative - because issuing a new release will not change the underlying use cases.
  • Shall be capable of reuse without change to support the Data Model of any SIF locale (US, AU, UK).
  • Shall extend and not compromise existing data security
  • Shall not weaken "out of the box" interoperability between SIF-conformant applications.
  • Shall extend the SIF standard into new developer environments such as REST without fragmenting SIF adopters into separate non-interoperable communities.


 In the next posting we will explore the extended set of high level infrastructure functionality which will be supported by SIF v3.0, in accordance with the above commandments.

Tuesday, July 3, 2012

SIF Implementation Specification (US) 3.0 release


The SIF Implementation Specification (US) 3.0 is on target for delivery at the end of the year, and it represents the first new “major version” release of the SIF standard since 2007.  If there was a one sentence description of the value proposition driving the development of this release, it would be “Putting CEDS on the wire”. 

Essentially this means:
  • The SIF Association will continue to be a core contributor to the ongoing Common Educational Data Standards (CEDS) development effort led by the US Department of Education.
  • The entire CEDS Logical Data Model spanning Pre-K, K-12 (already in 95%+ agreement with SIF) and Higher Education, will be mapped into a SIF Physical Data Model spanning the entire US Educational Domain, and additional SIF elements will also be incorporated.
  • The resultant “CEDS-driven Physical Data Model” will be placed on-the-wire by defining the underlying formats, context, exchange patterns, transport options and security constraints (i.e. the infrastructure) over which information in compliance with this model can be exchanged.
The end result is a standard that provides interoperability between all SIF-conformant educational applications exchanging CEDS-defined data elements to provide and support state and district-wide solutions.  In addition, much of the effort to achieve these objectives will be focused on developing the underlying tools and techniques required to map any locale-specific Logical Data Model to a Physical Data Model.  The resulting artefacts should be reusable in affiliated SIF standard development efforts undertaken by the other SIF locales (AU and UK) in support of their own Logical Data Models.

Keep your eye on this blog over the coming months, where we will be discussing several aspects of this release in more detail prior to its official release towards the end of the year.

Thursday, May 24, 2012

SIF US v2.6: Extensions for Staff Assignment and Evaluation


With the pending release of SIF US v2.6 next month (it is currently half way through 30 day member review) it seemed timely to present a few more of the functionality highlights contained in this featureful version.  In this posting we will focus on how various federal requirements and initiatives related to Staffing are supported, by looking at two SIF object types introduced with this release.

Staff Section Assignment is a “junction object” which links a Staff member to a Section, and contains information about that relationship.  Probably the most important element is the Teacher of Record (TOR) value which assigns a responsibility or credit percentage for a specific portion of a student's learning to a particular teacher or set of teachers.  It can also be used to track whether or not a student comes into contact with a teacher or group of teachers, and the extent of that contact.
Besides the obvious instructional benefits, many teacher accountability systems utilize this kind of information to evaluate the effectiveness of teachers within the context of the subject being taught and the assessments used to measure student progress. TOR was specifically designed to support Race to the Top and existing APPR (Annual Professional Performance Review) regulations from various states. 

Staff Evaluation Info standardizes the reporting of additional APPR information. It includes elements containing Staff, School and Evaluator ID, Bargaining Unit, type of staff evaluation, evaluation milestone dates (from pre to post conference), evaluation tool(s) used, and a section by section breakdown of the evaluation results (name, score and scale) and the final recommendation.