Skip to content

Reflecting on Learning Analytics and SoLAR

The Learning Analytics and Knowledge conference (LAK16) is happening this week in Edinburgh. I unfortunately, due to existing travel and other commitments, am not in attendance.

I have great hope for the learning analytics field as one that will provide significant research for learning and help us move past naive quantitative and qualitative assessments of research and knowledge. I see LA as a bricolage of skills, techniques, and academic/practitioner domains. It is a multi-faceted approach of learning exploration and one where anyone with a stake in the future of learning can find an amenable conversation and place to research.

Since I am missing LAK16, and feeling nostalgic, I want to share my reflections of how LAK and the Society for Learning Analytics Research (SoLAR) became the influential agencies that they now are in learning research. Any movement has multiple voices and narratives so my account here is narrow at best. I am candid in some of my comments below, detailing a few failed relationships and initiatives. If anyone reading this feels I have not been fair, please comment. Alternatively, if you have views to share that broaden my attempt to capture this particular history, please add them below.

How we got started
On March 14, 2010, I sent the following email to a few folks in my network (Alec Couros, Stephen Downes, Dave Cormier, Grainne Conole, David Wiley, Phil Long, Clarence Fisher, Tony Hirst, and Martin Weller. A few didn’t respond and those that joined didn’t stay involved, with the exception of Phil):

As more learning activities occur online, learners produce growing amounts of data. All that data cries out to be parsed, analyzed, interrogated, tortured, and visualized. The data being generated could provide valuable insight into teaching and learning practices. Over the last few years, I’ve been promoting data visualization as an important trend in understanding learners, the learning process, and as an indicator of possible interventions.

Would you be interested in participating in a discussion on educational analytics (process, methods, technologies)? I imagine we could start this online with a few elluminate meetings, but I think a f2f gathering later this year (Edmonton is lovely, you know) would be useful. (Clarence, Alec, and I tackled this topic about three years ago, but we didn’t manage to push it much beyond a concept and a blog :) ).

At the same time, I sent an email to colleagues in TEKRI (Rory McGreal, Kinshuk, and Dragan Gasevic) asking if this could be supported by Athabasca University. Dragan promptly replied stating that “I can say that most of the things we are doing with semantic technologies are pretty much related to analytics and I would be quite interest in such an event”. Then he told me that my plan for a conference in fall 2010 were completely unrealistic asking “[who] would be a potential participant? How we can get any audience in December?”.

Dragan and Shane Dawson, who I connected with through a comment on this blog, are two critical connections and eventually friends. Except Shane. He is mean and has relationship issues. SoLAR would not exist without their involvement. Another important connect was Ryan Baker. Ryan started the International Educational Datamining Society a few years earlier. The fact that Ryan was willing to assist in the formation of a possibly competing organization speaks volumes about his desire to have rich scientific discourse. We ended up publishing an article in LAK12 about collaboration and engagement between our fields.

LAK11
Organization was slow plodding for the first LAK conference. We built out our steering committee (defined by anyone who agreed to join) to include Erik Duval, Simon Buckingham Shum, and Caroline Haythornthwaite). We set up a Google group at the end of March on Education Analytics. The bulk of the planning for the first conference happened in that Google Group. By the end of June, I had seen the light of Dragan’s wisdom and agreed to move the conference to 2011. The LAK11 conference was held in Banff, Alberta in March. Important to note that we paid $500 for that logo. It should have come with a hit of acid.

The financials of any first event are critical. There is always risk. I’ve had events fail that cost a fair bit of money – a social media conference that I ran in Edmonton was a pleasant financial failure. For LAK11, we received financial support from Athabasca University, CEIT (University of Queensland), Kaplan, D2L, and the Gates Foundation. We generated a profit of ~$10k and that was forwarded to the organizers of LAK12 (Shane Dawson) to help seed the next conference. We didn’t have a formal organization to share in the expenses so each organizer for the first several years had to bear the financial risk. Paying past success forward made things easier for the next event. Leading up to LAK14, we were legally organized as SoLAR and took on the financial risk for local organizers.

Finding a publisher
In order to improve the scholarly profile of the conference, we pursued formal affiliation with a publisher. For many academics in Europe and Latin America, this was important in order to receive funding for travel. Dragan made numerous attempts to get Springer’s LNCS volume affiliation for the conference. The LNAI affiliation ended up being the avenue that we were suggested to pursue. Dragan put in the application on September 11, 2010. Springer stonewalled us at great length. We finally received confirmation that they would publish on July 17, 2011. Needless to say, as a professional organization, we did not want to work with a partner where that type of delay was considered acceptable. We were fortunate to connect with ACM and our first proceedings were published with them. Simon Buckingham Shum and Dragan were critical in securing this relationship, and in many ways for the academic rigour now found in LAK. I have been appropriately criticized by top researchers like Ryan Baker that the conference proceedings aren’t open. It was a decision that we made to broaden, oddly enough, access to travel funds to researchers from other countries.

My momma don’t like you
Not everyone was a fan of the idea of learning analytics. As this discussion thread on Martin Weller’s blog post reveals, there were voices of doubt around the idea of learning analytics:

Wish you luck in pursuing this Next Greatest Thing. Maybe next year’s can include the words “Mobile” “Emergent” and “Open” to broaden its hipness even further…really, really, really have been trying very hard not to make any comments since I first saw this announced early in 2010. I mean REALLY hard, because that comment above doesn’t even start to capture the amount of bullshit this smells like to me. But I am sure it will be a smashing success, a new field will have been invented, and my suspicions that there is no ‘there there’ even more unfounded. History will surely side with you George, of that I have little doubt.

Some of these doubts have become reality due to a techno-centric view of analytics, as is often captured by Audrey Watters. Interestingly, one of my first interviews on LA was with Audrey when she was writing for O’Reilly. The field has sometimes moved distressingly close to solutionism and Audrey has rightly turned toward criticism. We need more criticism of the field – both from researchers and practitioners and I find people like Audrey who are bluntly honest are essential to progressing as a research domain.

LAK11

Leading up to LAK11, I organized a LA MOOC (haha, MOOCs were so cool back then). This served as an opportunity to get people onto the same page regarding LA and to broaden possible attendance to the conference. LAK11 was fairly small with about 100+ people in attendance.

About two days before LAK11, I sent out an email stating:

We are expecting a week of nice weather – beautiful for strolling around Banff and enjoying the amazing scenery. Weather in the Canadian Rockies can be a bit temperamental, so it’s advised to pack clothing for the possibility of some chilly days.

Well, I lied. We were expecting -2C. We got -35C. Freaking cold for those of you that haven’t experienced it before. Also, it generated exceptionally high attendance rates as few people wanted to be outside.

The conference agenda (here) reveals the significant contributions of early attendees. While my first email to colleagues included my blogging network (Stephen, Alec, Dave, Martin) the LAK conference itself resulted in me engaging with a largely new social network disconnected from much of what I had been doing with connectivism and MOOCs, though there were points of overlap. In many ways, I see both MOOCs and LA as an extension of my thinking on connectivism as my more recent focus on the social, affective, and whole person aspects of learning.

Expanding and Growing

Following LAK, we spent some time organizing and getting our act together about what we had created. Over time it became clear that we needed an umbrella organization – one that was research centric – to guide and develop the field. On Oct 2, I sent the following email to our education analytics Google Group. I include the bulk of it as it reflects our transition to SoLAR – the Society for Learning Analytics Research.

With interest continuing to grow in learning analytics – at institutional, government, and now entrepreneurial levels – some type of organization of our shared activities might be helpful.

Based on the sentiment expressed at the post-LAK11 meeting on developing a group or governing body for learning analytics, a few of us have been working on forming such an organization. In the process, I’ve had the opportunity to meet and chat with several SC members (Erik Duval, Dragan Gasevic, Simon Buckingham-Shum) on different organizational structures that might serve as a model. We’ve done enough organizing work, we think, to open the discussion to a broader audience…namely the LAK SC (that’s you).

We’ve decided on Society for Learning Analytics Research (SoLAR) as a name for our organization. The term was coined by Simon Buckingham-Shum (program co-chair, LAK12). Obviously, we would like to invite existing LAK conference steering committee members to be a part of it. Are you interested in transferring your SC role to SoLAR? If so, please provide an image of your lovely head as well as a preferred link to your site/blog/work and a few sentences about how awesome you are.

We have also reserved the domain name: solaresearch.org for our society.

We envision SoLAR as an umbrella group that runs the LAK conference, engages in collaborative research, work with research students, scholar exchange, applies for grants, provides access for researchers to broader skill sets than they might have on their team, produces publications, etc. SoLAR is expected to be an international society/network where learning analytics researchers can connect, collaborate, and amplify their work. It is possible that SoLAR may occasionally provide feedback on policy details as states and provinces adopt LA. Maybe that’s a bit too blue sky…

Over the next few months, various documents will be drafted, including a charter, mission, and decision making process for SoLAR. For example, how do we elect officials? How do we decide where the conference will be held next year? etc. We (currently: Shane, Simon, Dragan, Caroline, John (Campbell), and myself) recommend that an interim SoLAR leadership board – the group just listed – be tasked with developing those documents and sharing with the SoLAR steering committee for comment and approval. Once this interim leadership has completed its organizing work, we will then open the process to democratic elections based on SC and society membership. We haven’t yet determined the criteria for being a SoLAR member (fees? attend a conference? invite only?) or how long SC members serve. Currently we are a self-organized group. Everyone is here either by an invite or expressing interest. Laying a clear, democratic, foundation now will help to position SoLAR as a strong advocate for learning analytics in education.

LAK12 was a tremendous success. Shane was a spectacular host. It became clear to us that interest was high in LA as a research activity and practice space. We arranged a meeting following the conference where we brought in ~50 representatives from funding agencies, corporations, and government officials. The intent was to discuss how LA might evolve as a field, what was needed to broaden impact, and how grant and foundation funding might assist in improving the impact of work.

Following LAK12, SoLAR engaged in a series of initiatives to improve the sharing of research and increase support for faculty entering the field. We had spent time in late 2011 discussing a journal, but didn’t get much traction on this until 2012. In early April, Dragan and Simon had put together an overview of the journal theme and it was approved by SoLAR executive and announced at LAK12. Dragan, Simon, and Phil were the first editors. Simon stepped down shortly after it started and Shane stepped in. Shane and Dragan have been the main drivers of the Journal of Learning Analytics.

A mess of other activities were started during this time including workshops at HICCS (organized by Dan Suthers, Caroline Haythornthwaite, and Alyssa Wise), Storms – local workshops, Flares – regional conferences, events affiliated with other academic organizations such as learning sciences. Basically, we were putting out many shoots to connect with as many academics and practitioners as possible.

One activity that continues to be highly successful is the Learning Analytics Summer Institute (LASI). In August of 2012, I sent Roy Pea from Stanford an email asking if he’d be interested in joining SoLAR in organizing a summer institute. We felt the Stanford affiliation signalled a good opportunity for SoLAR. Roy agreed and we started organizing the first event.

Roy and I didn’t connect well. Roy felt I was too impatient. I was pushing too hard to get things organized. Academic timelines always give me a rash. We managed to secure significant funding from the Gates Foundation and the first LASI was a success, in no small part do to Roy’s organizing efforts. After LASI, we decided to move the institute to different locations annually – a perspective that I strongly pushed as I didn’t want LASI to be affiliated with only one school. Due to my head bumping with Roy and suggestions to host the next LASI elsewhere (Harvard it turned out), I was written out of the final learning analytics report that he produced for the Gates Foundation on LASI. Academics are complex people!

A list of LASI, Flare, and LAK events can be found here.

Getting the finances right

Follow LAK11, we started exploring university subscriptions to SoLAR. This was informed by Shane’s thinking on paying an annual fee to be involved in groups such as NMC or EDUCAUSE. We set up a series of “Founding Universities”, each committing about $10k to be founding members. This served to be a prudent decision as it gave us a base of funds to use for growing our membership and hosting outreach events. Our doctoral seminars, for example, are funded and supported by these subscriptions.

We had strong corporate support as well with organizations like D2L, Oracle, Intel, Instructure, McGraw-Hill, and others providing support for the conferences and summer institutes. Corporate support has proven to be valuable in running successful conferences and enabling student opportunities. We decided to stay away from sponsored keynotes so as to ensure academic integrity of our conferences. I continue to be disappointed that we have been largely unable to get support from pure LA companies such as Civitas and education research arms of companies such as SAS. The students that we graduate grow the field. LA companies benefit from field growth. Or at least that’s my logic.

The founding members and current institutional partners are listed here. Each one has been central to our success.

Enter Grace
Grace Lynch joined SoLAR work in 2012. During LASI at Stanford, she pitched the idea of hiring someone to do administrative and organizing work with SoLAR. Up to that point, we were run by academics devoting their time. The work load was increasing. And those who know me also know my attention for detail is somewhat, um, varied. Hiring Grace was the best decision that I made in SoLAR. She was able to get us organized, financially and administratively. The success of SoLAR and LAK and LASI events is due to her effort. I frequently hear from others who first attend a SoLAR event about how impressed they are with the professionalism and organization. That’s Grace’s doing.

Engaging with with big ideas
During LAK11, we expressed our goals as an association:

Advances in knowledge modeling and representation, the semantic web, data mining, analytics, and open data form a foundation for new models of knowledge development and analysis. The technical complexity of this nascent field is paralleled by a transition within the full spectrum of learning (education, work place learning, informal learning) to social, networked learning. These technical, pedagogical, and social domains must be brought into dialogue with each other to ensure that interventions and organizational systems serve the needs of all stakeholders.

In order to serve multiple stakeholders, beyond LAK/LASI/Journal, we also held leadership summits and produced reports such as Improving the Quality and Productivity of the Higher Education Sector: Policy and Strategy for Systems-Level Deployment of Learning Analytics.

We have also been active in helping to shape the direction of the field by advocating for open learning analytics – a project that is still ongoing.

Losing Erik Duval
When one’s personal and professional worlds come together, as they often due in long term deep collaborative relationships, individual pain becomes community pain. Erik Duval, a keynote speaker at our first LAK conference, passed away earlier this year. He shared his courageous struggle on his blog. Reading the Twitter stream from LAK16, I am encouraged to see that SoLAR leadership has set up a scholarship in his honour. His contributions to LA as a discipline are tremendous. But as a friend and human being, his contributions to people and students are even more substantive. You are missed Erik. Thank you for modelling what it means to be an academic and a person of passion and integrity.

What I am most proud of
LAK is a unique conference and SoLAR is a special organization. I have never worked with such open, non-ego, “we’re in it because we care”, people in my life. I wish that future leadership also has the pleasure of experiencing this collegial and collaborative spirit. Our strengths as a community are in the diversity of our membership. This diversity is reflected in global representation and academic disciplines. As a society, we have better gender diversity than what is found in many technical fields. It is not where it should be yet. And the progress that we have made is due to the advocacy of Caroline Haythornthwaite and Stephanie Teasley. The current executive is a reflection of that diversity.

What’s next
At LAK15, I stepped down as founding president of SoLAR. I felt like it was time to go – I’ve seen too many fields where a personality becomes too large for the health of the field. We’ve always emphasized that SoLAR should be a welcoming space where individuals from different disciplines and research interests can find a place to play, to work, to connect. In order for this to happen, fluid processes for getting opinionated people out and new ideas in is important!

My attention is now primarily focused on two areas: developing LA as a field in China and increasing the sophistication of data collection. Recent visits to China, Tsinghua University and Beijing Normal University as well as an Intel LA event in Hangzhou in fall, have made it clear to me that LA is robust, active, and sophisticated in China. In many of the projects and products that I’ve seen, they’re well ahead of where the current state of publishing in English suggests that we are. In conversations with colleagues at Tsinghua, we have agreed to make the development of a research network and academic community in China a key priority.

Secondly, at LINK Research Lab, we have turned our research attention to wearables and ambient computing. As I stated in my keynote at LAK12, increasing and improving the scope and quality of data collection is needed in order to improve the sophistication of our work as a field. Physiological and contextual data will assist in advancing the field, as will a greater focus on social and affective aspects of learning. Cognition is only one aspect of learning. As a consequence, focus on affective, social, meta-cognitive, and process and strategy is required. To get there, we need better, broader data.

Well, that’s my reflection how we got here with LA and SoLAR. What have I missed?

3 Comments

  1. Hi George
    I sometimes regret not staying involved with LA, it was one of those things that had to give.
    I’ve become quite interested in the ‘recent’ history of ed tech. We’ve been through a lot of changes in 20 years and often don’t notice or document these. So I found this a really useful record of how something gets started and then gains sustainability. Regardless of what people think about ed tech, it’s a very useful account of how to do that for any field of interest.
    Future historians will thank you.

    Friday, April 29, 2016 at 12:40 am | Permalink
  2. I was told earlier in the week that folks like Canadians because we are irreverent. So here is my view from the fringes:

    In 2010 I became interested in the field, participated in the MOOC and attended LAK2011. The diversity of the views and the deep understanding of learning that the presenters and attendees brought with them was inspiring.

    Between 2011 and 2014 I worked in the space of data-supported learning design in a corporate setting but did not attend the LAK conference. When I returned to LAK in 2015, I was surprised in what appeared to by a dramatic shift in its focus and the composition of the attendees towards the field of computer science. By 2016, attendees pushed back at Paul Kirschner’s suggestion that an understanding of learning, or least the inclusion of a learning scientist, is important. (kudos to the organizers for including him as a keynote)

    I submitted papers in 2014 and 2016 and the review feedback has been intriguing. The most recent feedback included, “From my reading, it seemed as if the authors simply used existing course/content/SBL usage data to identify areas of improvement. I like the content of the presentation, I just question its applicability to LAK.”

    I’d argue that if improving content quality (and results) is no longer the primary purpose of the field, then it might be time to question the applicability of LAK to the fields of learning and teaching.

    My gut and anecdotal experience tell me that there might be some interesting patterns and stories bigger than my own that ought to be told. And when I am looking to flesh out a story, I often turn to the data.

    We have seen selected bits of data: Number of overall submissions, demographics and topics of accepted authors and papers.

    But there are far richer data sources available if they were made openly available: The participant lists and (anonymous) LAK Easy Chair data from LAK2011-2016. Raw data just waiting to be analyzed.

    We could also then combine that information with other open data (with a special shout out to Doug Clow’s amazingly complete LAK blog notes (dougclow.org)) to further enhance the narrative.

    In short, apply what we propose to do to students to ourselves. There might be a couple of insightful stories in that data. We might just learn a thing or two…

    Saturday, April 30, 2016 at 5:38 am | Permalink
  3. Thanks for this helpful history George. Working with you, and seeing SoLAR develop has been a truly fantastic experience!

    Simon

    Tuesday, May 3, 2016 at 4:08 pm | Permalink