Using Data from an LMS Data Portal

From E-Learning Faculty Modules



Contents

Module Summary

In 2016, Instructure (the makers of the popular Canvas LMS) made a data portal available to its mainline customers, and in late 2016, K-State activated its access and conducted an early review of the available data. This module summarizes what sorts of data may be found in an instance of an LMS and its particular data portal, how to handle the data, some types of answerable questions from LMS data portal data, some analytical techniques that may be applied to the data, and some practical ways to use the extracted insights.


CanvasDataCredentialPortal.jpg

Takeaways

Learners will...

  • review what an LMS data portal is and how it relates to other data sources in an LMS (in this case, Instructure’s Canvas)
  • explore the types of information in an LMS data portal and the structure of a data table from an LMS data portal
  • consider ways to handle LMS data portal data (which is “big(gish)”) and some of the potential askable questions from such data
  • review a number of computational and human methods to query the LMS data portal data
  • brainstorm practical ways to improve teaching and learning from LMS data portal data and some other types of practical insights from LMS data portal data


Module Pretest

1. What is a “data portal” for a learning management system? What is a “data dictionary” (or “schema docs”)? How is an LMS data portal related to the front end of an LMS? The built-in reports for instructors on an LMS? The admin-accessible pre-packaged reports on an LMS?

2. What types of information are included in an LMS data portal? What is the structure of a data table from an LMS data portal?

3. What are some ways to handle the big(gish) data from an LMS data portal? What are some askable questions? What are some practical ways to approach the use of data from an LMS data portal?

4. What are some computational methods to query the data from an LMS data portal? What is sentiment analysis? What is time-to-event analysis? What is machine learning, and how can this approach be used to identify patterns in the online learning data? What is human close reading of text-based information on an LMS data portal, and how can these result in research-based insights?

5. What are some practical ways to improve teaching and learning from LMS data portal data? What are some other practical uses of the insights accessible from an LMS data portal?


Main Contents

1. What is a “data portal” for a learning management system? What is a “data dictionary” (or “schema docs”)? How is an LMS data portal related to the front end of an LMS? The built-in reports for instructors on an LMS? The admin-accessible pre-packaged reports on an LMS?

A learning management system (LMS) is a technology that enables a range of capabilities for online learning—the making of persistent role-based accounts, the delivery of teaching and learning contents, intercommunications, assessments, team-based projects, and other features. An LMS data portal is a space where some of the data created as a byproduct of an LMS’s functions is made available to users. In many cases, faculty roles are not enabled for the access to such data portal data, but administrator access is.

For this particular LMS data portal, the data is updated daily, and they are contained in 79 data tables which may be downloaded as a zipped folder of .gz files, which can be uncompressed using 7Zip.

On the front end of an LMS, users with varying levels of role-based access can take online courses, collaborate on projects, engage in automated learning, provide academic advisement, serve on hiring committees, and use the site in a variety of ways. On the back end of an LMS are limited-access sites with data such as the reports feature and the LMS data portal feature.

The faculty using the front end of the LMS have access to some pre-packaged data about their courses and their learners within those courses. Administrators have access to pre-packaged reports that are set up by semester terms (as time units). Administrators also have access to the LMS data portal. In combination, the built-in course-level reports, the admin-level reports, and the LMS data portal provide some level of information about the particular LMS instance, but these do not contain all levels of information available. These, even together, may be challenging in terms of forensic analysis usage.


2. What types of information are included in an LMS data portal? What is the structure of a data table from an LMS data portal?

Depending on the company / organization making the LMS available, there can be a wide range of data made available. For a full and up-to-date listing of the data table types, please refer to the publicly available [Canvas Data Portal Schema Docs 1.17.1 https://portal.inshosteddata.com/docs].

As noted, there are data tables about assignments, assignment groups, conversations (in-instance emails), course sections, course user interfaces, discussions, enrollments, external tool activations, group membership, modules, quizzes, role types, submissions, submission comments, users, wikis, wiki pages, and so on.

The structure of a data table in an LMS data portal, in this case, is a classic one with columns containing defined data types, and rows containing individual records. The columns are not labeled here, and users have to refer to the data dictionary / schema docs to understand more clearly what each column contains.


3. What are some ways to handle the big(gish) data from an LMS data portal? What are some askable questions? What are some practical ways to approach the use of data from an LMS data portal?


[“Wrangling Big Data in a Small Tech Ecosystem” http://scalar.usc.edu/works/c2c-digital-magazine-spring--summer-2017/wrangling-big-data-in-a-small-tech-ecosystem] provides some tips on how to handle the data.

The askable questions relate to the captured and reported information in the data portal. As noted earlier, it is easier to understand issues of proportionality of “states” of items (courses, assignments, users, etc.), time-based issues, and other descriptive insights, often at the macro level of the instance.

In terms of micro-level types of queries, these may be harder to assert in any solid way.


4. What are some computational methods to query the data from an LMS data portal? What is sentiment analysis? What is time-to-event analysis? What is machine learning, and how can this approach be used to identify patterns in the online learning data? What is human close reading of text-based information on an LMS data portal, and how can these result in research-based insights?

Various tools may be used to extract meaning from the data.

  • For example, sentiment analysis may be applied to the messaging (or titling) extracted from the LMS data portal.
  • Time-to-event analysis (“survival analysis”) may be applied to time-based data.
  • Machine-learning-based induced decision trees may be applied to properly structured data tables (albeit within certain size limits).
  • Human close-reading may be applied to text-based information for research-based insights (such as how LMS course shells are being for non-course applications based on names applied to some of the courses).

Such back-end data is best used with deep and accurate intimacy with the LMS and its uses at the local institution of higher education—so that the understandings may be achieved with finesse.


5. What are some practical ways to improve teaching and learning from LMS data portal data? What are some other practical uses of the insights accessible from an LMS data portal?\

From the LMS data portal data, it is possible to capture a sense of how the LMS is being used…and to conceptualize what the data is showing about how the LMS is being used in support of teaching and learning…and enabling some suggestions for how to improve this.

Some initial ideas may be seen in the following online poster [Using Large-Scale LMS Data Portal Data to Improve Teaching and Learning at K-State https://www.slideshare.net/ShalinHaiJew/poster-using-largescale-lms-data-portal-data-to-improve-teaching-and-learning-at-kstate].

Of course, ideas for how to improve teaching and learning have to be advanced with political sensitivity and finesse.

Examples

For more on this work, please see the Extra Resources links below.

How To

The basic steps to access the Canvas LMS data portal, in this K-State instance, is as follows:

1. Log in.

2. Select the “Admin” button in the left menu.

3. Identify which account one wishes to access.

4. Select the “Canvas Data Portal” link.

As for the more complex steps to handling the data and some of the analytic approaches, those are addressed in partially in [Wrangling Big Data in a Small Tech Ecosystem http://scalar.usc.edu/works/c2c-digital-magazine-spring--summer-2017/wrangling-big-data-in-a-small-tech-ecosystem].

Possible Pitfalls

As with any data source, proper use of data involves full knowledge of how the data was captured, how much noise is in the data, the designed usage of the data, and so on. This will require a deep understanding of the LMS (in terms of the front end use), a close reading of the schema docs / data dictionary, a deep exploration of the unwieldy big data, and mastery of a range of tools and methods to query the data. The LMS data portal involves some information only, not all. When explored as “flat files,” because the primary and foreign keys are not easily accessible, there are limits in querying across data tables (except by time).

Based on some initial explorations, most of what is extractable here involves summary data based on a holistic sense of the LMS instance…or a particular slice-in-time of the LMS instance.

The LMS data portal data may not be as assertable as possible, in a close-in way. If there are questions about a particular learner or group of learners, those may not be accessible without the files reconstituted in a SQL database (it seems). Even the admin-accessed reports have their limits of assertability in terms of a micro-assessed way.

The LMS data portal data seems to be extracted data that is collected as a byproduct of the functioning of the LMS.

The data collected may differ between instances, and

Another risk is that some of the data tables include personally identifiable information (PII)—of all users of the LMS instance—and these have to be handled with care (without any leakage of the private information).

Outcomes data such as grades are not included in the LMS data portal data (as of this writing), so this limits some forms of outcomes analysis. Similarly, competency outcomes are not included either.

Module Post-Test

1. What is a “data portal” for a learning management system? What is a “data dictionary” (or “schema docs”)? How is an LMS data portal related to the front end of an LMS? The built-in reports for instructors on an LMS? The admin-accessible pre-packaged reports on an LMS?

2. What types of information are included in an LMS data portal? What is the structure of a data table from an LMS data portal?

3. What are some ways to handle the big(gish) data from an LMS data portal? What are some askable questions? What are some practical ways to approach the use of data from an LMS data portal?

4. What are some computational methods to query the data from an LMS data portal? What is sentiment analysis? What is time-to-event analysis? What is machine learning, and how can this approach be used to identify patterns in the online learning data? What is human close reading of text-based information on an LMS data portal, and how can these result in research-based insights?

5. What are some practical ways to improve teaching and learning from LMS data portal data? What are some other practical uses of the insights accessible from an LMS data portal?


References

Hai-Jew, S. (2017, Spring-Summer). Wrangling Big Data in a Small Tech Ecosystem. C2C Digital Magazine. http://scalar.usc.edu/works/c2c-digital-magazine-spring--summer-2017/wrangling-big-data-in-a-small-tech-ecosystem

Schema Docs – Canvas Data Portal. (Oct. 2017). https://portal.inshosteddata.com/docs.

Extra Resources

Hai-Jew, S. (2017, Spring-Summer). Wrangling Big Data in a Small Tech Ecosystem. C2C Digital Magazine. http://scalar.usc.edu/works/c2c-digital-magazine-spring--summer-2017/wrangling-big-data-in-a-small-tech-ecosystem

Hai-Jew, S. (2017, Aug. 5). Using Large-scale LMS Data Portal Data to Improve Teaching and Learning (at K-State). International Symposium on Innovative Teaching and Learning and its Application to Different Disciplines. Sept. 26 – 27. Digital Poster Session. Kansas State University. https://www.slideshare.net/ShalinHaiJew/using-largescale-lms-data-portal-data-to-improve-teaching-and-learning-at-kstate.

Hai-Jew, S. (2017, Aug. 16). Using Decision Trees to Analyze Online Learning Data. International Symposium on Innovative Teaching and Learning and its Application to Different Disciplines. Sept. 26 – 27, 2017. Kansas State University. https://www.slideshare.net/ShalinHaiJew/using-decision-trees-to-analyze-online-learning-data.

The K-State Online Canvas LMS Data Portal and Five Years of Activated Third Party Apps. (2017, Sept. 22). Have a Byte! https://spark.adobe.com/page/PQRxrknjH2nxp/.

Schema Docs – Canvas Data Portal. (Oct. 2017). https://portal.inshosteddata.com/docs.