Andrea Resmini & Bertil Carlsson
Working paper, draft as of December 10 2013.
Part of the Proceedings of Relating Systems Thinking & Design 2013, Oslo School of Architecture and Design, Oslo, October 9-11 2013.
We initially applied this methodology to two courses originally to be given in the spring semester of the first year of the Master: IT Project Management in International Settings, and Methods to Evaluate IT Ventures, each granting 7,5 ECTS credits.
The two courses were refactored and tailored to offer, instead of a plain traditional approach to project management and innovation, a comprehensive overview of the complexity hidden behind the ideation, development and deployment of innovative information systems-based solutions, and the basis of a practice-oriented holistic methodology the students could use to approach these messy or wicked problems in their future roles as managers or designers. Attention was paid to avoid presenting the courses and their content as linear experiences: especially in the Methods to Evaluate IT Ventures course, we emphasized how the lectures were introducing parallel concepts and views, not any kind of “skill development track”, and that these configured a messy system of sometimes conflicting methods and theories rather than a progression.
For example, when introducing Free and Open Source Software, a new subject for many of the students, we carefully avoided placing it simply as either the better choice over proprietary software on economic or ethical grounds, or a substandard alternative on business terms. We rather suggested that the adoption of any such solution would have to be weighted through a systems thinking lens, carefully considering not only business requirements, as it usually happens, but the ecosystem in which it would manifest its effects and create new or change existing feedback loops.
Additionally, the structure of the courses themselves was modified to allow for two distinct, parallel learning tracks: a formal system of traditional lectures, leading up to an individual written exam meant to test a student's proficiency with the course literature and theory, and a hands-on group project, leading up to a final oral presentation with public question time in front of the whole class. Credits were split across the two moments, with a .5 imbalance favoring the written exam due to its individual nature, but with an explicit acknowledgement that group work, regardless of this imbalance, was a foundational part of the courses and could not be taken lightly or superficially without compromising one's final grades or results.
As the courses were being reconfigured, we decided not to rely on any one book for literature, but rather assemble the necessary theory through articles and book parts we could handpick. A result of this process was the introduction of a slightly modified literature seminar as an intermediate verification step within the course on Methods. Literature seminars are a common feature of courses in Informatics at JIBS, used to introduce students to reading and understanding academic writing: we gave it a system angle, using group conversations as a way to cover all or most of the theoretical subjects we introduced in the course.
We divided articles and book chapters in mandatory readings, and secondary readings. Then we pulled most articles we considered secondary from the mandatory reading list, leaving there only the foundational materials students would be tested for in the written examination. These pulled papers were used to constitute a small heterogeneous library of academic and industry articles clustered around the subject matters of the courses, from which we allowed students to choose freely, with the only rule being that as a group they needed to first cover all areas before any single member could double up and choose a paper from a cluster already chosen by another group member. Groups then orally presented their readings for public discussion and filed short individual reports.
Issues of free-riding or passenger behavior (Race, 2007) within the groups were discussed up front, and the introduction of formal contracts (Gallow, 2011) to be signed by the group members to stipulate reciprocal obligations were briefly considered but discarded in favor of a much more personable approach based on mutual recognition, teacher-student and student-student, that the learning goals of the courses would prove to be unattainable if not through everyone's commitment.
In all, free-riding proved to be a non-issue, and constant monitoring not necessary once the groups got started. Expressly asked about group performance and group dynamics, all project managers reported no real problems beyond the occasional, and thoroughly manageable, cultural or interpersonal misunderstanding throughout the duration of the courses.
We also took particular care to redress student's expectations of what failure meant within the boundaries of the courses, especially in relation to the hands-on projects: as for many of them group work, and specifically designerly (Cross, 2006) group work, would have been a novel activity, we constantly stressed that only through constant application, iteration, and reflection upon their own errors would they gain the insights upon which all and any evaluation would be based, decoupling learning errors from their final official results. We expressly devoted an initial class meeting to addressing this framing through gamestorming (Gray et al, 2010), by introducing the students to their group mates and the courses underlying philosophy of learning through the Marshmallow challenge (Wujec, 2010).
Learning by doing
The Marshmallow challenge is a low-tech group design exercise invented by Canadian innovator Tom Wujec, which “encourages teams to experience simple but profound lessons in collaboration, innovation and creativity” (Wujec, 2010). Groups are given 20 dry spaghetti, one meter of tape, one meter of string, and one marshmallow. In 18 minutes, they have to produce the tallest self-standing structure they can build, with the marshmallow perched on top. The challenge is largely set up so that mostly everyone fails, especially students in business schools, as Wujec, who has been running the challenge with a large variety of groups, from Fortune 500 executives and CEOs to children in kindergarden, explains in a TED talk.
The usual approach of students is to overthink things out before starting any actual construction, verbally engaging each other in lengthy conversations about strategy. As a result, they waste quite a lot of time, generally do not consider that the weight of the marshmallow radically alters the balance of whatever structure they have, and end up with either very short but standing spaghetti frames, or with tall but inevitably collapsing ones.
This is where we introduced the ideas that not only the one approach that works is the one little children would use, a constant iterate-until-it-works process, but that what they will do within the courses will be a mirror of the exercise itself: approach issues hands-on, try out many different and often wrong strategies, reflect, obtain deeper knowledge on both the subject matter and the process, apply it, iterate.
Even more importantly at this early stage, we reassured them that, as in the Marshmallow challenge, making mistakes is a part of learning, especially when facing systemic, complex issues. Final scores in the courses would be a reflection of their learning process, certainly built up through this practical, sizable process of trial and error, but definitely not hampered by failures along the way. Such a premise would probably be superfluous in a design school, but proved to be absolutely necessary in the context of a business school (Sherwood, 2002) where students more rarely encounter this type of learning by doing (Cross, 2010) strategies.
The original plan for practical activities within the courses accounted also for engaging the classes with brief systems thinking exercises (Booth Sweeney & Meadows, 2008) whenever possible, but this was soon abandoned because of time related constraints in favor of more focus on their own projects.
Entrepreneurship, ownership, renewal
Students at JIBS are formally encouraged to progress through their curriculum by always thinking entrepreneurial as well as responsible. The approach we adopted in each of the two courses then, perfectly in line with the holistic framing of the learning outcomes as they were introduced in the modified syllabi, was to challenge the students to both seize any entrepreneurial opportunities that arose during the process as well as think of possible consequences of any changes they were either suggesting or introducing for the larger ecosystem, the goal being of creating meaningful connections to JIBS' strategic vision for higher education.
Both courses are strategically placed within JIBS three main focus areas of ownership, renewal and entrepreneurship, and hence not only our aim was to encourage students to identify entrepreneurial opportunities within the problem spaces introduced within the course itself, as mentioned, but we wanted them to approach their group project exercise preferably with renewal as a general goal, and responsibility as the underlying constraint.
Improvements to the services would have to happen responsibly, considering outcomes and the impact that all proposed changes would have on the system as a whole. Finally, we wanted them to especially pay attention to the human actors, and set them up to map all stakeholders, whether taking part in the production, delivery or consumption processes belonging to the service or system, and resolve all ownership sides of the venture.
While still conceived as independent learning moments because of constraints in the organizational layout of the master program, both courses however adopted the same internal structure and strove to offer students a view over the design of information systems through the lens of multiple competing, diverging perspectives: the business model and ethical point of view of free and open source software (Lessig, 2010); the lenses of usability and user experience (Unger & Chandler, 2009); the attention to the co-production models of cross-channel and pervasive computing (Resmini & Rosati, 2011); and the structured approaches of enterprise information architecture (Guenther, 2013) and security. These were tied together through an overarching systems thinking approach that provided the theoretical framework for both the theoretical framing and the project work.
Structure of the courses
The courses were held in English and structured to provide the class of largely non-Swedish, non-European students with both the basics of theory necessary to understand and formalize a systems thinking approach, through the frontal lectures track, and with practical knowledge about tools and methods they could then apply to their project assignments to produce the required deliverables, through group-by-group tutoring and public classroom discussions.
Most activities within the courses were organized on a weekly basis, with the expectations of steady progress through the project tasks and deliverables and with the availability of both regular scheduled tutoring and brief on-demand assistance during normal working hours.
Globally, scheduled tutoring amounted to roughly 3-4 hours per week for all groups, while on-demand tutoring never exceeded 1-2 hours per week. Students relied mostly on scheduled meeting for the more intensive project-related questions and conversations, using impromptu requests for assistance mostly for practical, course-related questions connected to deadlines, administrative procedures, and personal issues.
All formal learning moments such as the public presentation of final results were left for the closing week in the courses, which ran for a length of 10 weeks before the exams.
While especially the course on Methods to Evaluate IT Ventures tried to emphasize the designerly aspects of dealing with information systems and information-based services in a cross-channel world (Resmini & Rosati, 2011), the courses took place within the context of a business school. For this reason, a business perspective was kept central to the projects, acting as an initial filtering layer for the students, and while the lectures connected subjects to JIBS' educational vision and explored in depth the different subjects and their connections, we took great care in keeping the ideas of plausibility and feasibility very present into the students' minds during project work. Was the proposal the group was pushing forth defensible from a business perspective? Were they considering ethics? Personal or societal consequences?
Examples were taken from the lectures and tied to the groups' current work on the projects: the concept of responsibility in action, for example, was illustrated through cases documenting successful adoption of free and open source software in public organizations with positive effects on governance and transparency. We introduced usability and user experience to convey the idea that in order to renew a product or service, not only do we need to change or increase any value proposition for potential customers, but we have to adopt a user-centered perspective. Ownership and empowerment were reconnected to the multiple trajectories of personal data and remediation in cross-channel and co-production (Tapscott & Williams, 2008).
Instead of having the students focus on understanding one single point of view, often identifiable in the process / project management perspective, the courses aimed at creating an understanding of the elements at play and of their reciprocal relationships. Students were constantly led towards adopting a reflective, iterative process, and encouraged to embody their temporary conclusions in communication artifacts to be shared and discussed.
On the basis of previous experiences with classes largely composed of international students, no voluntary pairing or grouping was allowed to avoid creating an uneven landscape of perfect strangers and tightly knit buddy teams. Groups were randomly assembled: this was also introduced as part of the learning process building up on the initial exercise with the Marshmallow Challenge, in that groups had to figure out their own working structure first and get to choose a project manager among themselves.
Once groups were settled and project managers in place, the students were presented with 5 larger areas of possible intervention, all connected to different aspects of the theoretical frameworks outlined in the courses and all supporting JIBS focus on entrepreneurship, ownership and renewal: public transportation systems, healthcare information systems, cross-border public services, multi-agent systems and social networks, educational solutions for children in primary school, and ambient or pervasive systems.
These were offered as binding suggestions: within the limits of the different areas, groups were required to formulate a verbal research proposal to investigate an existing service ecosystem that allowed them to successfully identify and circumscribe either specific issues or a problematic general situation, all the time providing reasons as to why they though it was worth exploring that resonated with the overall goals of the courses.
At this stage, groups largely functioned through verbal interactions. While we encouraged sketching and note-taking, and in general visual representation of group conversations, at all times, no formal requirements were in place for groups to provide us with written reports of any kind.
Once groups had worked their way through a definition of the boundaries of their intervention, clearly stating limitations, doubts, and exploratory questions, the process acquired an increased degree of formalism and we moved on to deliverables. These had to be provided to us in standardized A4-sized PDFs that had to be uploaded within given deadlines through our class management software. Again, we took care to explain that groups were not going to be judged based on graphic or visual design skills or proficiency, but on the clarity of their reflections and of the degree of confidence with which they successfully handled the systems thinking approach they were applying.
Groups would proceed through a diagrammatic representation of a typical customer journey for someone using the service or system, and finalize a business model representing the current status via the business model canvas (Osterwald & Pigneur, 2011). As a final step in this stage of their learning process, they were then requested to transition these findings into a desirable future state by applying Gharajedaghi’s system mapping framework (Gharajedaghi, 2011) of searching, mapping, and telling the story, and to adopt a user-centered approach (Unger & Chandler, 2009). The customer journey, the business model canvas, and the system mapping deliverables would all be carried over into the final oral presentation and discussion in front of the class.
Emphasis was placed at all times in clarifying how there could be no preordered right or wrong answers, but rather a varying degree of fittingness and utility depending on how they successfully (or unsuccessfully) set boundaries around their systems and managed to identify sufficiently specific research questions that could be answered within the limits of the courses themselves, both in terms of resources available, data obtainable, and time constraints.
Similarly, while basic deliverables (such as the business canvas, customer journey, a final written report detailing in full their exploratory process, and a slide deck to be used for the final presentation) where mandatory for all groups, and students were introduced to a number of different tools and methodologies for synthetic representation of knowledge artifacts, including personas and service blueprints, no particular requests or restrictions were posed on the groups as far as secondary deliverables were concerned, with the explicit goal of allowing each of them to develop their own personal representation of their problem space. This proved to be a challenge for some groups.
Public discussion of group work and written examination
At the end of the courses, groups were requested to present their work to the rest of the class in an allotted slot of twenty minutes, and to conduct a public critique aimed at reflecting on their project through the lenses of individual, organizational and societal indicators as they are commonly found in socio-economic or socio-technical (Bjiker, 1997) evaluations. For each of these we asked the students to formulate how their proposed future states would affect each indicator in respect to the problem areas. Groups presented with the help of a slide deck containing all mandatory deliverables, but with no given fixed template: again, it was made clear to them that visuals and graphics were not elements contributing to the assessment of their work. A public moderated discussion followed. Participation through questions was constant and well structured.
The written examination was carried out through a multiple-choice questionnaire structured through 20 questions (initially 15) of varying difficulty, each providing a varying amount of points (either 3, 5, or 7), for a total 100 points. Each and every question could only have one correct answer, and all of them could be traced back directly to either the lectures and their slide decks or to the mandatory readings for the courses.
We introduced systems thinking as an overarching approach for teaching students of the Master in IT Management and Informatics at Jönköping International Business School how to solve complex or wicked information systems problems through a designerly approach. We changed two courses in the Spring semester 2013 in order to provide the students with both a theoretical appreciation of the multiple and often conflicting points of view any complex system brings to the table, and with a practical hands-on training through project work where they could readily apply the tools and methodologies they were learning along the way. JIBS strategic vision of entrepreneurship, ownership and renewal shaped the way the group work was framed.
Formal and informal evaluation post facto showed that students appreciated the courses, and while the process is far from being concluded or finalized in any way, a few major preliminary take-aways can be drawn from our observations, the courses results, and the students' own comments.
- The students thoroughly enjoyed the courses, the format, the hands-on approach and appreciated the systems thinking perspective.
- The students founds the visual approach of some of the tools and methodologies adopted challenging. This was expected, but at times it resulted in increased pressure on the teaching team and in generalized requests for step-by-step guidance: a brainstorming session for using the business model canvas became a painstakingly slow and detailed explanation of every single action to be performed.
- Cultural differences played a large role in some of the difficulties some of the students experienced (Ladd & Ruby, 2010).
- The examination process for courses of this type, where students are evaluated through a weighted combination of individual and group performances, has to be rebooted since the classical academic project report methods (e.g. IMRAD) do not provide a full picture of what the project parts were about and of the impact they have. This will require research, because while a better and more suitable examination process is auspicable, conformity to the examination rules given by the Swedish regulatory bodies needs to be maintained.
- The free-flow framing proved to be somewhat problematic to a part of the students, who feared that the freedom they were exerting in placing boundaries around their problem space would directly translate to a difficult assessment and possibly unfair final grades as they were “missing the point”.
- More generally, the courses was perceived to offer no real boundaries, which implied that students were at loss as to what they would be judged upon. While this was intentional and deliberate, it made them feel insecure and was a major cause of delays and misunderstandings.
- A part of the students did not feel comfortable enough within an open ended process, and while they could understand the project goals well enough, they could not bring themselves to systemically intervene on their deliverables, nor connect the different points of view provided by the subject areas into a holistic view. Similarly, they had a hard time connecting these back to JIBS educational strategy. One of them stated that “a more traditional set of lectures centered around one single idea, with possibly mandatory individual readings covering the remaining topics, would have been preferable”.
As these courses were implemented and deployed in a period of months, failings were considered an integral part of the process and, after completion, informal meetings, supplementing the formal evaluation process all courses receive, were scheduled to better gauge the outcomes. Furthermore, we collected notes throughout all interactions with the classes, and regularly held teachers-only meetings to discuss them and assess whether any tuning was necessary.
Nonetheless, the courses scored in the high percentiles in all 15 standard survey questions used for evaluation at JIBS, with the general query “My overall impression is that the course was good” being scored 5-7 by a compound 70% of the surveyees (7 being “strongly agree”, 1 being “strongly disagree”), with no results below 4 for one course and one single result at 3 for the other.
All informal post-mortem assessments we conducted also agreed that the courses offered a liberating hands-on approach that allowed students to create connections between their work in the class, and messy situations they are confronted with when dealing with real-life projects. Similarly, students appreciated the possibility to turn their intuitions into visual representations: they simply could not do it well and felt frustrated.
It is our opinion that this could be solved in future iterations by introducing targeted lectures combining the theoretical foundations of the different lenses or approaches with hands-on workshops where the students get to work with the tools and reflect on them while still having easy access to a lecturer whenever questions should arise.
Incremental changes and future work
Because of the vast geographical pool from which students come to JIBS, because of cultural differences among them, and because of the multidisciplinary nature implied by teaching a Master in Informatics within a business school, we have been careful in generalizing any indications, either positive or negative, as they have been obtained from a single run affecting only two courses.
At the same time, the results can certainly be qualified as highly positive, and on the basis of both the findings we exposed here and of further planning carried out during the Summer of 2013, a few incremental adjustments have been already introduced to the courses which have been run as a second iteration in the first part of the Fall semester 2013. More changes are currently being considered for when the official syllabi for the courses will be reviewed in 2014.
The major changes introduced during the Fall of 2013 semester have been connected to the project work. Groups have been randomly created at the beginning of the academic year, and maintained throughout all courses given during the semester. Similarly, groups have been tasked to choose their projects early on, and have kept working on it from the different angles introduced by the four different courses involved in the process through a common systems thinking approach based again on Gharajedaghi's system mapping process.
It remains to be seen how the issues pertaining to the perceived absence of boundaries can be solved without resorting to a more traditional linear model.
Bjiker, W. E. (1997). Of Bicycles, Bakelites, and Bulbs – Toward a Theory of Sociotechnical Change. The MIT Press.
Booth Sweeney, L., and Meadows, D. (2008). The Systems Thinking Playbook. Sustainability Institute.
Cross, N. (2006). Designerly Ways of Knowing. Springer.
Gallow, D. (2011). Working Effectively with Small Groups. Teaching, Learning & Technology Center, Division of Undergraduate Education. University of California Irvine.
Gharajedaghi, J. (2011). Systems Thinking – Managing Chaos and Complexity. Morgan Kauffman.
Gray, D., Brown, S., and Macanufo, J. (2010). Gamestorming – A Playbook for Innovators, Rulebreakers, and Changemakers. O'Reilly.
Guenther, M. (2013). Intersection. Morgan Kauffman.
Ladd, P. D., and Ruby, R. (1999). Learning Style and Adjustment Issues of International Students. Journal of Education for Business. Vol. 74. No. 6.
Lessig, L. (2005). Free Culture. Penguin Books.
Meadows, D. H. (2008). Thinking in Systems – A Primer. Chelsea Green Publishing.
Osterwalder, A, and Pigneur, Y. (2010). Business Model Generation – A Handbook for Visionaries, Game Changers, and Challengers. Wiley.
Race, P. (2007). The Lecturer's Toolkit. Routledge.
Resmini, A., and Rosati, L. (2011). Pervasive Information Architecture – Designing Cross-channel User Experiences. Morgan Kauffman.
Sherwood, D. (2002). Seeing the Forest for the Trees – A Manager's Guide to Applying Systems Thinking. Nicholas Brealey Publishing.
Tapscott, D. and Williams, A. D. (2008). Wikinomics – How Mass Collaboration Changes Everything. Portfolio Hardcover.
Unger, R., & Chandler, C. (2009). A Project Guide to UX Design. New Riders.
Wujec, T. (2010). The Marshmallow Challenge. Available at http://marshmallowchallenge.com/Welcome.html. Last visited Nov 12 2013.