Dec 10 2013

Teaching systems

Teaching systems was written with Bertil Lindenfalk and presented at the October 2013 RSD2 Symposium, Oslo School of Architecture and Design (AHO), Oslo, Norway. This is a camera-ready draft. The original paper is available as part of the Relating Systems Thinking and Design 2013 Symposium Proceedings and in a revised version on Vol. 7, No. 3 of the peer-reviewed journal Formakademisk

Abstract

Information is going everywhere. It is bleeding out of the Internet and out of personal computers, and it is being embedded into the real world. Mobile devices, networked resources, and real-time systems are making our interactions with information constant and ubiquitous. Information is becoming pervasive, and products and services are becoming parts of larger systems, many of these emergent, complex information-based ecosystems where participants are co-producers and where relationships between elements, channels and touchpoints are messy and non-linear. Still, by and large, within the area of informatics and information systems we teach management and design as if they were linear processes where cause and effect are easily ascertained and a solution readily provided. Could we try something different? How would that work and what results could it produce in terms of both learning outcomes and student satisfaction? This paper details the approach we followed and the early results we achieved in introducing business and informatics students to entrepreneurship and innovation through a holistic approach in the 2-year Master in IT, Management and Innovation at Jönköping International Business School (JIBS), in Jönköping, Sweden.

Approach

We initially applied this methodology to two courses originally to be given in the autumn and spring semesters of the first year of the Master program: Project in Informatics (PI), and Methods to Evaluate IT Ventures (ME), each granting 7,5 ECTS credits.

The two courses were refactored and tailored to offer, instead of a plain traditional approach to project management and innovation, a comprehensive overview of the complexity hidden behind the ideation, development and deployment of innovative information systems-based solutions, and the basis of a practice-oriented holistic methodology the students could use to approach these messy or wicked problems in their future roles as managers or designers.

Traditionally, this is not the approach followed within the school, nor it is the approach followed for the teaching of business information systems (BIS) in informatics settings. Courses normally provide step-by-step unambiguous approaches that tend to break down structures through well-ordered, clearly laid out ontologies, hide the complexity and messiness of human decision-making, and do away with their very nature of socio-technical systems. Enterprises, services, and organizations are approached as isolated artifacts or collection of isolated artifacts that can be repurposed or refactored in the void, with little if no systemic consequences, and whose problems can be solved by thoroughly answering a definite set of carefully crafted initial requirements.

In their well-known book “Strategic Management of Information Systems” (2009), widely used as supporting material for courses on BIS innovation and in its fifth edition, Pearlson and Saunders for example introduce the Information Systems Strategy Triangle, a synthetic way to describe the interdependency of organizational, information, and business strategies within a company. While it can be argued that the framing is an inward-looking construction that totally neglects the reciprocal influences between a company and its environment, it is clearly a step towards a systemic view of enterprise strategy. Unfortunately, the triangle is introduced and quickly abandoned. All throughout the book clearly defined problem spaces are approached linearly and separately, as if governance has little or no ethic implications, or changes to the supply chain have no bearing on the company's own ecosystem: once requirements are clear enough, a solution will follow.

As such, the courses aimed at introducing first and foremost the idea that managing and designing BIS and BIS innovation is a systemic activity, moving the students away from a step-by-step, cause-and-effect, problem-and-solution mindset, and introducing them to treating business organizations, their environment, and their services as interacting parts of complex systems.

This was reflected in the educational structure of the classes: attention was paid to avoid presenting both the courses and their content as linear experiences. Especially in the ME course, we emphasized how the lectures were introducing parallel concepts and views and not one single “skill development track”, and that these configured a messy system of sometimes conflicting methods and theories rather than a continuous progression.

It is also important to note that we tried to establish a strong sense of connection between the two courses, both in terms of the general intended learning outcomes, and in terms of practical day-to-day activities: while still conceived as administratively independent learning moments, the two classes shared the subject chosen for the hands-on group work, which was then explored as a continuous problem space for the full length of the combined total weeks of the courses, adopted the same internal structure, and strove to offer students a view over the design of information systems through the lens of the same set of multiple, competing, diverging perspectives.

These were identified in a series of extended meetings between the teachers, the program director, and researchers in informatics. The goal was to present a multifaceted overview of BIS management and design that would reinforce our systemic approach, and allow for multiple entry points, technical, societal, experiential, into the problem space of service innovation. The final list of topics included the business model and ethical point of view of free and open source software (Lessig, 2010); the lenses of usability and user experience (Unger & Chandler, 2009); the attention to the co-production models of cross-channel and pervasive computing (Resmini & Rosati, 2011); and the structured approaches of enterprise information architecture (Guenther, 2013). These were tied together through an overarching systems thinking approach that provided the theoretical framework for both the course overall framing and the project work.

For example, when introducing Free and Open Source Software, a new subject for many of the students, we carefully avoided placing it simply as either the better choice over proprietary software on economic or ethical grounds, or a substandard alternative on business terms. We rather suggested that the adoption of any such solution would have to be weighted through a systems thinking lens, carefully considering not only business requirements, as it usually happens, but the ecosystem in which it would manifest its effects and create entirely new feedback loops, or change existing ones.

Additionally, the structure of the courses themselves was modified to allow for two distinct, parallel learning tracks: a formal system of traditional lectures, leading up to an individual written exam meant to test a student's proficiency with the course literature and theory, and a hands-on group project, leading up to a final oral presentation with public question time in front of the whole class.

Fig. 1 - The conceptual structure shared by the courses

Credits were split across the two moments, with a .5 imbalance favoring the written exam due to its individual nature, but with an explicit acknowledgement that group work, regardless of this imbalance, was a foundational part of the courses and could not be taken lightly or superficially without compromising one's final grades or results.

It must be stressed that because of administrative and educational constraints, either internal to the program or belonging to the overall framing of the business school, we could not completely dispose of all existing scaffolding and radically alter the courses mechanics throughout. This was for example the reason for retaining the written exam in the first place.

Literature and seminars

As the courses were being reconfigured, we decided not to rely on any one book for literature, but rather assemble the necessary theory through articles and book parts we could handpick. This allowed us to fine-tune our sources to the experimental nature of the courses, but also familiarized ourselves with the materials we would be teaching in those areas we were not individually comfortable. A result of this process was the introduction of a slightly modified literature seminar as an intermediate verification step within the ME course. Literature seminars are a common feature of courses in Informatics at JIBS, used to introduce students to reading and understanding academic writing: we gave it a systems angle, using group conversations as a way to cover all or most of the theoretical subjects we introduced in the course.

We divided articles and book chapters in mandatory readings, and secondary readings. Then we pulled most articles we considered secondary from the mandatory reading list, leaving there only the foundational materials students would be tested for in the written examination. These papers we pulled were used to constitute a small heterogeneous library of academic and industry articles clustered around the subject matters of the courses, from which we allowed students to choose freely, with the only rule being that as a group they needed to first cover all areas before any single member could double up and choose a paper from a cluster already chosen by another group member. Groups then orally presented their readings for public discussion and filed short individual reports.

Fig. 2 - The courses as a connected process and continuity through project work

We also took particular care to redress student's expectations of what failure meant within the boundaries of the courses, especially in relation to the hands-on projects: as for many of them group work, and specifically designerly (Cross, 2006) group work, would have been a novel activity, we constantly stressed that only through constant application, iteration, and reflection upon their own errors would they gain the insights upon which all and any evaluation would be based, decoupling learning errors from their final official results. This was a particularly hard fact to grasp for those of them used to identify the level of correctness of an exercise with either formal correspondence to quantitative teacher-set objectives, or with replicating content from literature. Business school students tend to conceptualize these as foundational parts of what their outcome in a course will be, both in terms of grades and learning.

As the varied multi-cultural environment added a secondary layer of complexity (Lewis, 2005), we initially setup a fairly extensive safety net consisting of scheduled tutoring and supervision hours, that proved to be both unnecessary and under-resourced. Unnecessary because individual needs for both tutoring and day-to-day supervision were largely solved in-group, with no request for teacher intervention, and under-resourced because, as we document later on, the major complaint that the students expressed post facto was that they thoroughly enjoyed the experience but, regardless of our continuous support, felt that at times they lacked the tools and the skills to communicate clearly in designerly terms.

To ease the class into what was for them an unconventional mindframe, we expressly decided to devote the initial class meeting to a gamestorming session (Gray et al, 2010), and introduced the students to their group mates and the courses' underlying philosophy of learning by means of the Marshmallow challenge (Wujec, 2010).

Learning by doing

Gamestorming techniques are often used to facilitate innovation and group thinking in business contexts. They usually involve a facilitator setting the stage for a structured, playful challenge that has to be solved in groups, very often encouraging divergent and disruptive thinking. As gamestorming sessions usually employ common low-fidelity design props such as post-its, markers, cut-outs. they work extremely well as bridge-constructs between the less structured design workshop and the more structured business brief.

The Marshmallow challenge is a low-tech group design exercise invented by Canadian innovator Tom Wujec, which “encourages teams to experience simple but profound lessons in collaboration, innovation and creativity” (Wujec, 2010). Groups are given 20 dry spaghetti, one meter of tape, one meter of string, and one marshmallow. In 18 minutes, they have to produce the tallest self-standing structure they can build, with the marshmallow perched on top. The challenge is largely set up so that mostly everyone fails, especially students in business schools, as Wujec, who has been running the challenge with a large variety of groups, from Fortune 500 executives and CEOs to children in kindergarden, explains in a TED talk.

The usual approach of students is to overthink things out before starting any actual construction, verbally engaging each other in lengthy conversations about strategy. These usually result in a straightforward process of negotiation over the one solution to the problem that the available expertise and personal biases seem to suggest is appropriate. Students discuss triangles, squares, the adhesive properties of tape, oscillating between complete confidence in their own rationalizations and a sense of total failure, but do very little with their hands and when they do, they start very late in the exercise.

As a result, they waste quite a lot of time, generally do not consider that the weight of the marshmallow radically alters the balance of whatever structure they have, and end up with either very short but standing spaghetti frames, or with tall but inevitably collapsing ones. This is where we introduced the ideas that not only the one approach that works is the one little children would use, a constant iterate-until-it-works process, but that what they will do within the courses will be a mirror of the exercise itself: approach issues hands-on, try out many different and often wrong strategies and conceptualizations, learn, reflect, obtain deeper knowledge on both the subject matter and the process, apply it, iterate. As much as any such process is one of the foundational bases of learning for students of design and could be considered common fare, this is not the case for business information systems students.

Even more importantly at this early stage, we reassured them that, as in the Marshmallow challenge, making mistakes is an unavoidable and necessary part of learning, especially when facing systemic, complex issues. Final scores in the courses would be a reflection of their own individual learning experience, certainly built up through a guided, practical, sizable process of trial and error, but definitely not hampered by failures along the way. Such a premise would again probably be superfluous in a design school, but proved to be absolutely necessary in the context of a business school (Sherwood, 2002) where students more rarely encounter this type of learning by doing (Cross, 2010) strategies.

The original plan for practical activities within the courses accounted also for engaging the classes with brief systems thinking exercises (Booth Sweeney & Meadows, 2008) at distinct breakpoints along the way, but this was soon abandoned in favor of having the students devote more hours to work on their own projects. Core concepts that were intended to be explored in the systems thinking exercises were simply covered through frontal lectures.

Entrepreneurship, ownership, renewal

Students at JIBS are formally encouraged to progress through their curriculum by always thinking entrepreneurial as well as responsible. The approach we adopted in each of the two courses then, perfectly in line with the holistic framing of the learning outcomes as they were introduced in the modified syllabi, was to challenge the students to both seize any entrepreneurial opportunities that arose during the process as well as think of possible consequences of any changes they were either suggesting or introducing for the larger ecosystem, the goal being of creating meaningful connections to JIBS' strategic vision for higher education and expand it through systems and design thinking.

Both courses are strategically placed within JIBS three main focus areas of ownership, renewal and entrepreneurship, and hence not only our aim was to encourage students to identify entrepreneurial opportunities within the problem spaces introduced within the course itself, as mentioned, but we wanted them to approach their group project exercise preferably with renewal as a general goal, and responsibility as the underlying constraint.

Group assignments and project work

On the basis of previous experiences with classes largely composed of international students, no voluntary pairing or grouping was allowed to avoid creating an uneven landscape of perfect strangers and tightly knit buddy teams (Race, 2007). Groups were randomly assembled: this was also introduced as part of the learning process building up on the initial exercise with the Marshmallow Challenge, in that groups had to figure out their own working structure first and get to choose a group leader / project manager among themselves.

Once groups were settled and project managers in place, the students were presented with 5 larger areas of possible intervention, all connected to different aspects of the theoretical frameworks outlined in the courses and all supporting JIBS focus on entrepreneurship, ownership and renewal: public transportation systems, healthcare information systems, cross-border public services, multi-agent systems and social networks, educational solutions for children in primary school, and ambient or pervasive systems. These were offered as binding suggestions: within the limits of the different areas, groups were required to formulate a verbal research proposal to investigate an existing service ecosystem that allowed them to successfully identify and circumscribe either specific issues or a problematic general situation, all the time providing reasons as to why they though it was worth exploring that resonated with the overall goals of the courses.

At this stage, groups largely functioned through verbal interactions. While we encouraged sketching and note-taking, and in general visual representation of group conversations, at all times, no formal requirements were in place for groups to provide us with written reports of any kind.

Once groups had worked their way through a definition of the boundaries of their intervention, clearly stating limitations, doubts, and exploratory questions, the process acquired an increased degree of formalism and we moved on to deliverables. These had to be provided to us in standardized A4-sized PDFs that had to be uploaded within given deadlines through our class management software. Again, we took care to explain that groups were not going to be judged based on graphic or visual design skills or proficiency, but on the clarity of their reflections and of the degree of confidence with which they successfully handled the systems thinking approach they were applying. They were also encouraged to experiment with any of the tools and methodologies we introduced through the lectures, to choose the ones they felt better communicated their work, and to reflect on their choices.

Groups moved on to surveys, interviews, and brief ethnographic studies, acquiring a limited but relevant amount of user data in respect to their system, but mostly a hands-on understanding of how a rather large line-up of stakeholders, from end users to part suppliers or service providers, was influencing, shaping, and constantly remediating the systems they were mapping.

Groups would proceed through a diagrammatic representation of a typical customer journey for someone using the service or system, and finalize a business model representing the current status via the business model canvas (Osterwald & Pigneur, 2011). As a final step in this stage of their learning process, they were then requested to transition these findings into a desirable future state by applying Gharajedaghi’s system mapping framework (Gharajedaghi, 2011) of searching, mapping, and telling the story, and to adopt a user-centered approach (Unger & Chandler, 2009). The customer journey, the business model canvas, and the system mapping deliverables would all be carried over into the final oral presentation and discussion in front of the class.

Again, no requirements were imposed on how these deliverables were to be rendered: emphasis was instead placed at all times in clarifying how there could be no preordered right or wrong answers, but rather a varying degree of fittingness and utility depending on how they successfully (or unsuccessfully) set boundaries around their systems and managed to identify sufficiently specific research questions that could be answered within the limits of the courses themselves, both in terms of resources available, data obtainable, and time constraints.

Similarly, while these basic deliverables (the business canvas, customer journey, a final written report detailing in full their exploratory process, and a slide deck to be used for the final presentation) where mandatory for all groups, and students were introduced to a number of different tools and methodologies for synthetic representation of knowledge artifacts, including personas and service blueprints, no particular requests or restrictions were posed on the groups as far as secondary deliverables were concerned, with the explicit goal of allowing each of them to develop their own personal representation of their problem space.

While this was introduced to simplify the student's work, since we could not assume any previous general familiarity with visualization or presentation techniques at this time in the program, this ample discretion in the way they could communicate their own results proved to be a challenge for some groups.

Improvements to the services chosen by the individual groups as their project space would have to happen responsibly, considering outcomes and the impact that all proposed changes would have on the system as a whole. Finally, we wanted them to especially pay attention to the human actors, and set them up to map all stakeholders, whether taking part in the production, delivery or consumption processes belonging to the service or system, and resolve all ownership sides of the venture.

Issues of free-riding or passenger behavior within the groups (Race, 2007) were discussed up front, and the introduction of formal contracts (Gallow, 2011) to be signed by the group members to stipulate reciprocal obligations was briefly considered but discarded in favor of a much more personable approach based on mutual recognition, teacher-student and student-student, that the learning goals of the courses would prove to be unattainable if not through everyone's commitment.

In all, free-riding proved to be a non-issue, and constant monitoring not necessary once the groups got started. Expressly asked about group performance and group dynamics, students reported no real problems beyond the occasional, and thoroughly manageable, cultural or interpersonal misunderstanding for the whole run of the courses.

Structure of the courses

The courses were held in English and structured to provide the class of largely non-Swedish, non-European students with both the basics of theory necessary to understand and formalize a systems thinking approach, through the frontal lectures track, and with practical knowledge about tools and methods they could then apply to their project assignments to produce the required deliverables, through group-by-group tutoring and public classroom discussions.

Most activities within the courses were organized on a weekly basis, with the expectations of steady progress through the project tasks and deliverables and with the availability of both regular scheduled tutoring and brief on-demand assistance during normal working hours.

Globally, scheduled tutoring amounted to roughly 2-3 hours per week for all groups, while on-demand tutoring never exceeded 1-2 hours per week. Students relied mostly on scheduled meeting for the more intensive project-related questions and conversations, using impromptu requests for assistance mostly for practical, course-related questions connected to deadlines, administrative procedures, and personal issues. As we mentioned early on, group engagement and student-to-student support made most direct teacher-student supervision unnecessary.

All formal learning moments such as the public presentation of final results were left for the closing week in the courses, which ran for a length of 10 consecutive weeks each before the exams.

While especially the ME course tried to emphasize the designerly aspects of dealing with information systems and information-based services in a cross-channel world (Resmini & Rosati, 2011), the courses took place within the context of a business school. For this reason, a business perspective was kept central to the projects, acting as an initial filtering layer for the students, and while the lectures connected subjects to JIBS' educational vision and explored in depth the different subjects and their connections, we took great care in keeping the ideas of plausibility and feasibility very present into the students' minds during project work. Was the proposal the group was pushing forth defensible from a business perspective? Were they considering ethics? Personal or societal consequences?

Examples were taken from the lectures and tied to the groups' current work on the projects: the concept of responsibility in action, for example, was illustrated through cases documenting successful adoption of free and open source software in public organizations with positive effects on governance and transparency. We introduced usability and user experience to convey the idea that in order to renew a product or service, not only do we need to change or increase any value proposition for potential customers, but we have to adopt a user-centered perspective. Ownership and empowerment were reconnected to the multiple trajectories of personal data and remediation in cross-channel and co-production (Tapscott & Williams, 2008).

Instead of having the students focus on understanding one single point of view, often identifiable within traditional teaching frameworks in the process / project management perspective, the courses aimed at creating an understanding of the elements at play and of their reciprocal relationships. Students were constantly led towards adopting a reflective, iterative process, and encouraged to embody their temporary conclusions in communication artifacts to be shared and discussed.

Public discussion of group work and written examination

At the end of the courses, groups were requested to present their work to the rest of the class in an allotted slot of twenty minutes, and to conduct a public critique aimed at reflecting on their project through the lenses of individual, organizational and societal indicators as they are commonly found in socio-economic or socio-technical (Bjiker, 1997) evaluations. For each of these we asked the students to formulate how their proposed future states would affect each indicator in respect to the problem areas. Groups presented with the help of a slide deck containing all mandatory deliverables, but with no given fixed template: again, it was made clear to them that visuals and graphics were not elements contributing to the assessment of their work. A public moderated discussion followed. Participation through questions was constant and well structured.

The written examination was carried out through a multiple-choice questionnaire structured through 20 questions (initially 15) of varying difficulty, each providing a varying amount of points (either 3, 5, or 7), for a total 100 points. Each and every question could only have one correct answer, and all of them could be traced back directly to either the lectures and their slide decks or to the mandatory readings for the courses.

As we had to retain a written exam as part of the evaluation of the student's learning process in the courses, explicitly linked with the teachers' ability to verify fluency with the courses' literature and theory, a multiple-choice questionnaire was chosen for two primary pedagogical reasons: while Jönköping International Business School requires a certified English proficiency equivalent to a minimum 575 points score in the TOEFL paper based test, past experiences with heterogeneous classes of international students showed us that huge variations in written as opposed to oral fluency are to be expected; furthermore, variations would have been difficult to assess at the time, as the courses ran as part of the initial year in the program, one of them in the autumn semester. As educationally our primary interest remained connected to the changes introduced with the practically-oriented project work, the multiple-choice examination supported our goal to allow students to demonstrate knowledge on the subject matters of the courses without turning their attention away from group engagement.

Conclusions

Products and services are becoming parts of larger systems, many of these emergent, complex information-based ecosystems where participants are co-producers and where relationships between elements, channels and touchpoints are messy and non-linear. This is not reflected in the teaching of management and design of business information systems, where linear chains of easily ascertained causes and effects and unambiguous requirements from which solutions can easily be deduced is still the mainstream educational and reference framework.

We introduced systems thinking as an overarching approach for teaching students of the Master in IT Management and Informatics at Jönköping International Business School how to solve complex or wicked information systems problems through a designerly approach.

We changed two courses in the autumn and spring semesters 2013 in order to provide the students with both a theoretical appreciation of the multiple and often conflicting points of view any complex system brings to the table, and with a practical hands-on training through project work where they could readily apply the tools and methodologies they were learning along the way. The courses were strategically and pedagogically linked and JIBS strategic vision of entrepreneurship, ownership and renewal shaped the way the group work was framed.

As these courses were implemented and deployed in a period of months, failings were considered an integral part of the process and, after completion, informal meetings, supplementing the formal evaluation process all courses receive, were scheduled to better gauge the outcomes. Furthermore, we collected notes throughout all interactions with the classes, and regularly held teachers-only meetings to discuss them and assess whether any tuning was necessary.

Both formal and informal evaluation post facto showed that students appreciated the courses, and while the process is far from being concluded or finalized in any way, a few major preliminary take-aways can be drawn from our observations, the courses results, and the students' own comments.

Nonetheless, the courses scored in the high percentiles in all 15 standard survey questions used for evaluation at JIBS, with the general query “My overall impression is that the course was good” being scored 5-7 by a compound 70% of the surveyees (7 being “strongly agree”, 1 being “strongly disagree”), with no results below 4 for one course and one single result at 3 for the other.

All informal post-mortem assessments we conducted also agreed that the courses offered a liberating hands-on approach that allowed students to create connections between their work in the class, and messy situations they are confronted with when dealing with real-life projects in the information systems domain. Similarly, students appreciated the possibility to turn their intuitions into visual representations: they simply could not do it well and felt frustrated.

It is our opinion that this could be solved in future iterations by introducing targeted lectures combining the theoretical foundations of the different lenses or approaches with hands-on workshops where the students get to work with the tools and reflect on them while still having easy access to a lecturer whenever questions should arise.

Incremental changes and future work

Because of the vast geographical pool from which students come to JIBS, because of cultural differences among them, and because of the multidisciplinary nature implied by teaching a Master in Informatics within a business school, we have been careful in generalizing any indications, either positive or negative, as they have been obtained from a single run affecting only two courses.

Even so, the results can certainly be qualified as highly positive, and on the basis of both the findings we exposed here and of further planning carried out during the Summer of 2013, a few incremental adjustments targeting some of the points exposed above have been already introduced to the courses which have been run as a second iteration in the first part of the Fall semester 2013.

The major changes introduced during the Fall of 2013 semester have been connected to the project work, in an effort to reduce the amount of friction generated by the simple problem of working with others in multi-talented, multi-cultural pools. Groups have been randomly created at the beginning of the academic year, and maintained throughout all courses given during the semester, not only the PI and ME courses, thus forcibly increasing the required amount of interactions between group members. Similarly, groups have been tasked to choose their projects early on, and have kept working on it from the different angles introduced by the now four different courses involved in the process through a common systems thinking approach directly based again on Gharajedaghi's system mapping process. More changes, that will also include a reassessment of the role and format of a written exam within the PI and ME courses, are currently being considered for when the official syllabi will be reviewed in late 2014.

It remains to be seen how the issues pertaining to the perceived absence of boundaries, which are in a way part of the process we want the students to be exposed to and also naturally exacerbated by their own unfamiliarity with the exploratory nature of a design approach and the non-linear patterns and mapping brought in by the systems thinking framing, can be solved without resorting to a more traditionally bounded educational model.

References


andrearesmini.com :: gjort är gjort
All content available under the CC BY NC SA license
2005-2017 Andrea Resmini