Devcathlon Mockup Version 2

Last week my software engineering class split into three teams and each produced a different proposed interface for our semester project - the Devcathlon software developement game. The different interfaces had many common elements, but also introduced new ideas and generated discussion on the assumptions and goals of the project. A major point that we settled on was that Devcathlon should be dependent and bundled as an optional tool in the Hackystat software metrics system. Though potentially limiting some flexibility, it simplifies features like registration, login verification, data collection, and creates a more consistent environment. From here we split up into three new group configurations to further streamline and rehashed the proposed features.

Planning Phase

For this round I worked with John Zhou and Scheller Sanchez on Mockup5. We had a brief discussion on the basic needs of our interface right after class, but did most of our planning in a meeting a couple of days later. After setting up our SVN repository and Hackystat project we worked on identifying and defining the various elements of the web interface. It would have to allow for several states depending on the type of user being served. Users could assume one or sometimes several roles such as anonymous spectators, general logged in users, team leaders, team members, match members, or administrators. Then there are several different categories of pages to be implemented. First there are the static pages that include the login, about Devcathlon/instructions, events, badge listing, and level listing pages. These pages are not expected to change often though certain settings can be modified by an administrator. The other pages include the personal user profile, team profile, and match pages which can all be changed as a result of user activities on the site, including adding comments, and as a part of ongoing games. There is also the Hall of Fame page which isn't directly modified by users, but is automatically changed as a result of Devcathlon activity.
While the static pages will look roughly the same when viewed by anyone, the user profile, team, and match pages will vary depending on the access level of the user. A logged-in user can edit their own profile page and set privacy levels to determine what details others see. On the team page a members would be able to leave messages for the team, and the team leader would see extra functionality to make and accept challenges. A match page allows members to request and verify untracked events like pair programming and team meetings. I also thought it'd be interesting to have two message areas: a playing field for members of all participating teams, and then another that is for the "cheering section" of the non-member crowd.
At our meeting we also brainstormed on the requirement of creating three new events, three new "badges", and a type of leveling system. The badges and leveling system came out of groups during version one of the mockup and showed potential to be an important concept for the project. The badges would signify permanent milestone markers for a user or group such as winning a certain number of games or consistently committing code that builds. A leveling system, akin to the javablackbelt.com notion would signify long term growth and experience gained by accumulating wins and events.

Implementation

For the mockup we would need to show each of these pages as they would be seen by each type of user. This should simply involve adding a various buttons or fields to the page according to the users status. In production this would be accomplished just by having varying division tags that would be generated as active or inactive portions of the page. I was principally responsible for creating the events listing page. It would have two possible states: one for the administrator which allows him to edit each point value and criteria field, and one for all other users who would simply see the current settings of the local Devcathlon instance. My goal was to create a page similar to the Devcathlon's Google Code page with a quick list of event names but also a summary of their point values. Each event would then link to a complete description further down the page. If viewed by an administrator there would be fields for each quantitative value in an event and a save button. Three new events we came up include monitoring classpath changes, seeing that the points are spread evenly across the team, and an event that looked at the larger Hackystat picture. My experience with changes made to the classpath file past a certain point means someone or everyone is confused. Adding a new tool is acceptable, but having to continually readjust variables seems to be a sign of problems. For the event point spread event, Devcathlon would look at all team member scores once a day, compute an average, and then award or deduct a point to the team if everyone wasn't within some range of the average, perhaps 20%. It could also simply award/deduct from members within/outside the range. Another event could look at overall Hackystat data and award a point for each team member if the most metrics are "in the green."

For badges we thought they should reward consistent good work. This would include badges for being the point leader for 5, 10, 20, etc matches.

A leveling system I thought of would use the ratio of good events to bad events to determine leveling up. The first level would be a "2-bit Developer" attained when the user reaches a ratio of 2 positive events for every negative event. From there as the user got better and meticulous with their programming the could become a "4-bit Developer", then 8-bit, 16-bit, and so on. I'd also want to add some balances such as a minimum amount of matches, days, commits, or some combination necessary for each level attained.

Devcathlon brainstorming

There are a number of different approaches that can be taken in the initial designing of Devcathlon that could lock it into specific uses and environments. Finding a balance between usability, flexibility, and consistency is quite the trick.

One big issue I think is the question of Devcathlon's breadth of users. Do we want Devcathlon to be a public website linking developers around the world? This brings of issues of scale, fair comparison of projects, and privacy issues. Or is it a self-contained system that can be set up on a private local network for a school or a company? I think at this point the later makes more sense as the target, but the former would and should be completely possible at release. This makes features such as Hall of Fame a lot easier to manage since such scores will make more sense in a smaller, consistent environment. Even on a larger scale though, scores could be sorted by fields such as number of project/team members, lines of code, and duration of the project. Or perhaps some sort of normalization of points/events per line of code per day.

Another concern is how closely Devcathlon should tie into the Hackystat software metrics tool that is currently planned as the point of entry for much of the data that goes into determining scores. I do not know what other software metrics tools there are out there, but my feeling is that to make Devcathlon friendly to a wide audience of developers we want to make it flexible as possible by keeping it untied for Hackystat. Instead there would need to be methods to hook Devcathlon into diverse systems and use the reporting system of the systems to gather data. Project managers could define the format of data along with the point system for a project.

I'm also concerned about information overload that ends up feeling like spam. Personally I generally really don't want a text message and email from Devcathlon, plus one from Hudson, whenever someone breaks the build. One message is probably sufficient for me. But others will want more. I think users should be able to choose. A project manager can set up Devcathlon with a default way to report events, including options for special text formatting and composition to impart a storyline feel to a message. Ultimately though, users should have their own settings for event reporting tools and frequency.

Basic events should be pretty easy to set up in a project. Minimal non-empirical events such as commits or a broken build can be simply set with a point value and optional reporting mechanisms. Other empirical data such as dev time are only slightly more complicated. But I'd also want it to be possible to make custom events of much higher complexity. Allow multiple data variables and formulas to be combined to reward points. Allow time and date ranges, repeatability limits, number of occurrences, thresholds, milestones, etc. Perhaps you want to set up increasing returns on coverage. Going from 40% to 50% coverage could give the same number of points as going from 85% to 90% as developers eek out those tricker tests on multiple boolean paths and exception handling. These tools would allow advanced project administrators to tailor Devcathlon to their environment and goals.

I like to think that even in a private installation of Devcathlon, all users would be able to create projects. A user's profile management page could have a list of links to projects they are associated with. It could be sorted by activity, creation date, or some other criteria. Each project could also have certain buttons or links associated with them such as "Accept Invite" or if you are a project owner "Manage Project" which is of course where point settings are at (changeable during the project?), team and member management, and other options. You wouldn't even have to be a developer in the project, a likely scenario as a project manager. Both these type of pages are private "management" pages accessible only by certain people. There would be similar "viewing" pages that reflect much of this information but are read only. You might even want to have a private public page such as making the information on a project visible only to people actually involved in the project, but they still wouldn't be able to edit it without owner access. There would also be static public pages with information about the principles of Devcathlon and instructions on use. These could of course be customized for a local installation of the game.

These are just some issues and ideas for Devcathlon, and I'm sure more will come up as we dive into it. I hope to help create a system that is very flexible and accommodating to diverse users, but keeping it simple enough to quickly setup as a novice user. In the mean time, the Devcathlon development team has to keep the code itself from becoming too complicated and inflexible and dooming it to be a violation of our third goal of open source software of allowing an outside developer to modify and extend the source to fit their needs.

Devcathlon difficulties

These last few days were supposed to involve designing and implementing a mock-up prototype for the proposed Devcathlon system's user interface. Unfortunately I had a lot of difficulty wrapping my head around the scope of the project, as well as organizing my schedule in order to get the needed hours in. While things seemed like they were off to a good start, my contribution to the current mock-up is very minimal. All of the files in the mockup1 directory were done by my other team members, Anthony Du and Daniel Arakaki.

Design meetings

Initially I felt that the team was off to a good start, though I was a bit confused in our initial direction. I couldn't seem to understand whether we needed to be creating our own implementation of rules, or following those initially discussed on our ICS 414 discussion. Nevertheless, our team met on Wednesday and Friday and went over a number of use cases and implemented several pages based on some sample html. The basic design we came up together and individually called for at least the following pages:

• Home page with login, registration, help, and public viewing links
• Registration page collecting user name, email information, privacy options
• A private personal profile page to manage settings, view associated projects and teams, and start new projects
• A public personal profile page where other users can view selected information
• A private project profile page where a project manager or "gamemaster" can add members, setup teams, set privacy settings, choose events to track and their point values, create custom events, award special points, and mark a match as complete
• A public (read only) project profile for public users to view information on a project
• A page to define customized events that aren't within the scope of the basic event tracking
• A scoreboard page with a listing of current and completed projects
• Links on the main scoreboard page would lead to project scoreboard page would list scores be broken down by teams
• Team links would list scores by individual members, which would link to the member profile pages
• Pages with information and links to help users get acquainted with Devcathlon

We got several of these pieces done as a group on Friday and broke apart some of the remaining tasks to be worked on, however I never managed to make any progress on creating a custom event page or working out a good system of help pages.

Points unawarded

To help get a feel for what Devcathlon is all about, we were supposed to track our development using Hackystat and simple notes. However this definitely didn't go as planned, at least for me. My recent computer reformat left me with a lot of trouble even getting Eclipse back to where it was to do any sort of tracking through that sensor. In the end I didn't do any commits personally, so a mountain of negative points there. As a team, we didn't define and use any issue tracking either. As a mock-up, there wasn't a whole lot to track as tests, coverage, builds, coupling, and complexity really weren't involved. We had a couple of team meetings, of which I have various notes, but not what I would really call minutes.

Conclusions

Devcathlon is off to a rather rocky start for me, mostly as a result of personal confusion and difficulties. Unfortunately my team also suffers as a result of this, for which I definitely feel bad for letting them down. Definitely hoping to figure things out soon and get things back on track. I know I've got lots of ideas for the project, safely written in my always-present notebook. I just need to work on getting them off of paper and into the public domain!

Productive gaming: what if your boss wanted you to play games at work?

If you asked high school and college computer science students what they want to do, I'd wager that a great number, probably a lot more than the industry can support, have at one time or another said, "I want to make video games." Of those, many don't know just what is involved, and very few will go into that multi-billion dollar industry and get that chance. I certainly wouldn't say no to the right opportunity to join in. When I started at my currently place of employment as a student programmer, I had to learn an "ancient" language I'd never heard of before: M or MUMPS. After some light reading, the first thing I started work on to get comfortable with the language was to create a text- and keyboard-based version of "minesweeper" using M. Random mine placement, recursive safe-spot clearing, boundry limits, and all that good stuff. Though I never got it to what I would brand as "production ready," I learned a lot and my co-workers were impressed. To me, engineering is often about solving puzzles, and solving puzzles to create a puzzle game fit me well.

Devcathlon

Now I'm getting another chance to create a game system, but on a larger and more serious scale. As part of the second semester in my software engineering course at UH Manoa my professor, Dr. Philip Johnson, has proposed the creation of a produtivity game called Devcathlon, inspired by the developement game and a Hudson continuous integration plugin. The idea is to link the Devcathlon a game system into the Hackystat monitoring system and award (or take away) individual and team points based on good (and bad) software development practices such as creating unit testing, committing verified code, doing code reviews, and other principles. The goal is to create a game system that is motivation, fun, and productive. It would have foster both competition and collaboration, be fair, be challenging and rewarding, and be fun without being distracting. To achieve these goals requires knowledge of just what aspects of games can be used to create such an environment. Games draw from many different areas to create an enjoyable experience, and people will vary wildly over what whether a game is or isn't fun, as will their reasons. Some of the most widely known games are the most simple: go, chess, etc. No storyline or character developement is necessary. The areas of gameplay are simple grid layouts. They are international, unhindered by borders, linguistics, expensive materials, physical limitations, and copyrights. Go and chess have no element of luck beyond perhaps deciding who goes first. As mentioned in this article, they are fun because of the challenge, experience, and satisfaction gained from playing with a closely matched opponent. Each move opens up thousands (millions? trillions?) of others. Other wildly popular games with relatively simple rules and materials yet a vast possibility of moves, such as poker in America and else where, incorporate the possibility of more opponents, an element of luck and pyscological warfare, as well as real risks and rewards. However for other people, and especially with modern games, it takes a much more complex game system to hold their attention. Some of these elements are mentioned in this article. These games often have storylines, expansive visual and audio effects, complex character development, goal and reward driven design, and anywhere from zero to extensive interaction with others. These are generally geared toward entertainment, and occasionally learning, while games geared toward real world productivity are a bit more rare. Some of the best known such games would be categorized as "icebreakers" and "team-building exercises." The main challenge is usually not the stated game goals, but rather to break people out of their normal routine and spend time (aka money) now to create a better work environment for the future. Productivity games are about the people and product, not the game. "These games don't need winners as much as they need players," says Ross Smith. He also mentions score boards a number of times. The score board becomes the avatars of the players, marking their location on the path toward the goal.

Issues

To develop the Devcathlon system presents a number of technical, balancing, and even ethical challenges. While I'll gloss over the technical issues for now, I definitely want to address the other two at this time.

Balancing in games goes on in many different ways in an effort to keep the rules fair to players without being boring. In roleplaying games, stats are balanced so that a certain player does not have a constant advantage over other players. It generally a question of what one activity or ability is worth in comparison to another. Is breaking the build worse than submitting code with no unit testing? If it is worse, by how much? Depends on the nature of the break? I find this balancing issue interesting because I did a similar algorithm for work. In order to identify a person using their name, gender, date of birth, address, and phone number (about as many data points as tracked by Hackystat) I helped develop and test a weighted point based system. The weights weren't simply that a name was more points than gender since it's far more specific. There were many different conditions that would change other points. For example, if comparing two people shows they have the same address, the first name of the people becomes more important than the last name. How many-points-more-important took a lot of testing. While this kind of medical record algorithm is definitely more important to get right than the Devcathlon system, it shows how complex the systems can get to achieve the right result.

An issue that also jumped out to me was more along ethical lines. To be successful, a group using Devcathlon has to make all of their team a part of the reporting. Some people will be more reluctant. Some will be more distracted or nervous that their performance is being tracked in an additional way. Then there is the possibility of cheating, boosting, questional negotions ("Hey, Good Friend Joe, since you don't care much, can you break your team's build, my team's trying to get ahead") and so on. There is also the issue of instant notifications and reporting. I think this has to be controlled from two ends. Administrators of a particular game can control how often notifications are sent out, but players should also be able to only get notifications as often as they want, and of the type they want. Checking your score 20 times a day could be distracting. The system should update constantly, but have customizable updates to the display ranging from real-time to semi-hourly to end of day/week/project. The truely public boards also have to be scrutinized since development projects can differ widely. For example a hall of fame scoreboard could have categories such as team size(eg 5, 20, 100 people), lines of code (eg 10000, 50000, etc), development time frame (2 weeks, 2 years), and so on so that comparisons can be made appropriately. Addressing and testing such issues allow the game to be more widely used and enjoyed.