Lumberyard is Amazon’s AAA game engine built with integrations into AWS and Twitch. Lumberyard was initially released in April of 2016 and has had incremental releases every 6 weeks since then. The Lumberyard Editor is a fork of CryTek’s CryEngine and has been and is being re-built for modern, scalable game development.
I was the lone designer working on the level editing pieces for the Lumberyard Editor. I led the UX work, producing several major deliverables and presenting these to the stakeholders between April 2016 and July 2017.
I began working on the Level Editor in May of 2016 when it became clear that we needed to completely re-work the way that CryEngine let users make games. I worked closely with one PM and a team of several developers throughout the course of the project, driving the overall vision and breaking down that vision into incremental pieces the team could develop within their sprints.
My first act as lead designer on the Level Editor was holding a Design Sprint with the principal engineers who had architected many of the underlying systems we would build our new Level Editor on top of. Other members of the Sprint Team included 2 other designers and our PM. Throughout the week, we were able to uncover technical challenges, identify risks and concerns, and generate some initial concepts for testing.
From there, I picked up where we left off and began designing initial wireframes for an MVP of the Level Editor. Lumberyard had releases every 6 weeks, so many of our initial goals were getting incremental changes to the existing Level Editor into the release build. I worked together with the PM to plan out a phased approach to our design solutions. The first two phases could be done without any large architectural re-workings other than the ones that were already in place, with the third phase requiring a significant overhaul of the entire Editor architecture. The third phase was 90% of the way to our end goal and we scheduled a final fourth phase for a polishing pass to integrate additional customer feedback and solidify the final UI design for the Editor.
Our plans were generated based on a large pool of customer information that was generated from sources such as feature requests from internal customers, forum posts from external users, customer interviews with internal and external customers, contextual interviews with team members as they built out levels, and some initial baseline usability studies done on the early builds of Lumberyard. The PM and I prioritized the feedback relevant to the Level Editor and began working toward an iterative approach to a better Editor experience.
I had been wireframing solutions as we were developing the phased plan and throughout the course of a few months, the PM and I aligned on many high-level goals of where we wanted the Editor to end up. Once I finished the first pass of wireframes, I ran them by a few users in both 1-on-1 settings and focus group settings and got some high-level feedback from them. I took that feedback and integrated it back into my second pass of wireframes and from there, I created the first of several interactive prototypes of the Level Editor for usability testing. For our first test, we kept it simple and tested a workflow of creating a simple object in the scene and creating a template out of it. The workflows for doing this in the new Level Editor were significantly different than creating objects in CryEngine, so it was critical we gathered feedback early and often in order to create a solid foundation for our new Level Editor.
From our first prototype, we gathered a slew of useful information from users which was taken back and applied directly into the Level Editor. Our engineering team was not very keen on using customer data to dictate our path forward at the time, so for several months, incremental development froze as engineering opposed the customer feedback. At this point, it was challenging to make any progress because even though key stakeholders agreed with the direction, the development team did not agree and refused to build what was being designed. At this point, I split my time working both on tactical, incremental changes and future, vision-driving changes in order to help show a vision of where the Level Editor would eventually be going. This helped to sell both stakeholders and the dev team on our direction as they were able to more clearly see where we were headed and buy into it.
As engineering continued to pushback, I continued my normal design process and was iterating on designs to help inform the future state of the Editor. The PM and I were still sticking to our original phased plan, so I continued down that path to begin illustrating the future of the Editor. I took the feedback we had received from the initial usability study and integrated it into the design. Additional feedback was also received from the internal games team via bi-weekly focus groups, contextual interviews, and numerous brainstorming sessions I held with individuals on the local games teams. I finished an updated pass of the wireframes and began working on finalizing a pattern library of UI guidelines I had been working on in parallel sine the beginning of the project. With the newfound user feedback and a clear direction forward, I began iterating on visual and interaction paradigms to serve as the foundational patterns for the entire Editor. Just as I started this work, a large focus shift from upper management came as we needed to prepare a build to impress customers for GDC.
With GDC 2017 approaching, our team came under the gun to deliver a significant experience update to the Editor for creating gameplay elements. Leadership’s goal forced the team down a path of having to deliver something for the GDC deadline and through this time, I was able to change some of the core features of the existing Level Editor after a large amount of pushback from engineering. I held a few meetings with the core team showing the updated wireframes and most critical user feedback and from there, we began working on the GDC build of the Level Editor. Myself and the team demo’d the updated GDC build on the show floor and it was met with high praise from customers. The features I designed were some of the most critical pain points for customers building games, and their ability to see the features working live in a real demo environment was critical to their understanding of the capabilities of the Engine. Unfortunately, with only 8-10 weeks of development time prior to GDC and so many large, fundamental features to build, the GDC build was hacked together for the show and was riddled with compromises and instability.
After GDC, 12 weeks of development work went into stabilizing the build. During this time, I conducted a second round of usability testing with more finalized mockups that detailed out the remaining core pieces of the Level Editor experience. I put together a research plan and recruited users internally for the test and conducted the second usability test in a lab nearby. The feedback was overwhelmingly positive and users were able to quickly and efficiently complete most tasks without any problems. That being said, there was still a handful of feedback from our customers, which was presented to the team in a debrief two days after the final usability participant. With the data from the study, I designed some proposed alternatives based on the customer feedback and presented the design suggestions alongside each piece of customer feedback in our debrief. This allowed the team to see how we take customer feedback and apply it directly to our designs and got a lot of positive buy-in from the developers.
Today, the team is hard at work developing the pieces of the Level Editor that have been designed based on our second usability test and has been able to successfully deliver incremental changes to the Level Editor every 6 weeks. The process is very slow and the benefits have just recently been showing up as more users are getting a hold of newer versions of the Editor that utilize the updated Level Editor presented at GDC. Currently, the team has been able to close over 1,200 bugs and feature requests and has effectively replaced 80% of the original CryEngine workflows for creating game objects. Through our research, we have discovered that the new workflows save users ~20% of their time once they are up and running and we expect that number to increase as we continue to fill in functionality gaps in the Level Editor and supporting Editors in the months to come.
View Phase 1 Interactive Prototype (Amazon Internal Only)
Download Phase 1 Moderator Guide and Task List (Needed to understand Phase 1 Prototpe)
View Phase 1 Usability Findings
View Phase 1 Stakeholder Review Presentation
Many other deliverables available upon request.