TYPO3 NEOS – BACKEND USABILITY TESTS

TYPO3 Neos is the new open source CMS developed by the TYPO3 community. With many years of experience working with TYPO3 and other systems, the Neos team set itself the goal of creating the next generation CMS. They wanted to develop a system that is user-friendly, with easy learning and minimal training.

Have these objectives been achieved? The slogan on the Neos website is: “You do not need training to edit content in Neos – you already know how to do it. You just click on the part you want to edit – and start.”  Is this true?

So let’s do some tests!

To get a closer look and to get to know more about the usability of Neos for typical users, we invited 11 people to do some backend tests. Some already had experience with CMS-s. Others less so and worked with CMS for the first time. We prepared a test page on the 1.2.2 version (http://neos-usability.creativetest7.de/), a simple tutorial for the basic elements in the Neos backend and 6 tasks at different levels of difficulty.

Initial tutorial for editors

Here we present a description of each task and the lessons learned by our team based on the way editors tried to deal with the tasks. We will also share general comments and ratings about Neos made by our testers.

Our testers:

  1. Basia (administration, HR, marketing) – knows WordPress
  2. Magdalena (software tester) – without knowledge of CMS
  3. Kamil (marketing)- without the knowledge of CMS
  4. Lukasz (programmer) – without the knowledge of CMS
  5. Ania (fitness instructor) – without knowledge of CMS
  6. Karol (front-end developer) – knows  WordPress, SmugMug
  7. Agnieszka (software tester) – without the knowledge of CMS
  8. Anna (marketing) – without knowledge of CMS
  9. Olga (an academic) – knows Joomla
  10. Basia (social researcher, shopkeeping) – knows her online shop
  11. Darek (front-end developer) – knows  WordPress

Tests were carried out in two formats:

  • 6 live sessions which tracked users’ reactions and observed how they worked in the Neos backend. Where the testers encountered difficulties in performing the tasks, we gave them some help regarding editing and building new content in Neos.
  • 5 online session – editors were asked to perform tasks without any help from the tutor. Three online testers completed all tasks in the online session. The fourth editor performed tasks 1 and 6 and partially finished task 3. Another editor was able to do only the first task without any further help.

Tasks, problems and results

Task 1: Edit content.

On the Home page, correct a typo and add new content: e-mail address in the footer.

The aim of this task was to check the easy-use level of inline editing. None of the testers had problems with changing or adding new content. Some of the testers needed more time to get used to  the backend. The phrase “you already know how to do it” is borne out by the experiences of users with editing texts in the Neos backend. One of our tester was really surprised that it was so easy – she expected something more complicated.

“It is not so discouraging as I thought it would be”

The ability to view and edit the page in its frontend view form, is well appreciated by editors. The vast majority of customers can be intimidated or discouraged by administrative panels in which the user has to construct a site using multiple blocks/modules and sometimes cannot see the results of his/her changes instantly. In the case of Neos backend we have the content canvas view that mirros our site and it is quite natural for beginners. After the first task one of our editors said:  “It is not so discouraging as I thought it would be.”

We also observed that at the beginning, users preferred to click on the site menu and logo instead of the navigation panel.

Task 2: Change structure of elements.

Reorganise the content in the columns. The testers had to move some elements(video, text, image) between three columns and change the properties of the parent element to get different column sizes.

hipo-cs-neos-tests-usability-tests-by-creativestyle-web-agency-for-your-online-shop-application
Columns, wrappers and content collections

In the second task, we wanted to find out how people coped with moving elements around in a more complex structure. Our experience shows that users can get a bit confused if there are more elements, especially when the structure, purpose or rationale is not intelligible from the point of view of the editor (eg. the necessity of placing floating elements into wrappers).

cs-creativestyle-neos-tests-cms-open-source-inspector-1


Nodes that can be unnecessary in the editors’ opinion are marked in red

Some of our testers did not have any problems with the proper arrangement of the nodes and described the task as easy. Others had problems with moving the boxes into the content collections, then nodes disappeared and testers were confused. It was also a bit difficult to find out that the elements in the columns did not have the fixed width individually, but took the width of the parent element – Three Column Content.

Editors want to change properties and drag and drop elements on the content canvas

The vast majority of testers at first tried to find a way to change the width of the nodes such as image, video or text directly on the content canvas instead of changing the column. However it could be only a matter of lack of more detailed tutorial at the beginning. Every user from the live sessions at first tried to drag and drop elements to change the order. However after some explanation some of them also approved the idea of dragging and dropping inside the left panel or even found this feature by themselves. At this stage, we noticed that many testers used the structure tree  – probably based on a similar experience with folders. Editors were comparing the levels to find out where the given element was placed in the hierarchy and they also had no problems with closing or opening the parent element to get a better view of the content or entire structure.

When preparing tutorials for editors, it is a good idea to include detailed info about the necessity of inserting content into content collections or wrapping parent elements. It is important for the developers to explain carefully to editors their way of thinking while building the site structure: for example to show the right order of elements and nesting. Users may experience problems with some elements like content collections or wrappers because those elements are not visible on the page.

Task 3: Add a new slide.

The third task was as following: on the Home page after the third slide, add a new slide (fourth consecutive) with the supplied data (header, text, image).
The goal of the task was to examine the procedure of adding new elements. The majority of testers coped well with the implementation of the above command. However very often they did not remember about the tabs in the right inspector, which resulted in the problem with adding and editing images. Our experience shows that it is very frequent problem for editors on the initial stage of their work with the Neos backend. Therefore, in tutorials for editors, grouping properties in tabs should be clearly explained. Testers also had problems with closing the media panel – but the close button will be added in the new version of Neos. Some users forgot also about necessity to accept changes (apply) or to publish them – the procedure for publication should be also explained in details.

It is also important to remeber about placeholders for texts – without them users are really confused. The “Discard changes” feature is also important for many users – they feel safer if they know how it can be done.

neos-usability-tests-by-creativestyle-cms-opensource

Adding a new slider in the Neos backend

Task 4 – add parent element and some new elements inside it             

The task was to add a new section with three boxes. The editors picked up a hint to use a node ‘Box Wrapper/ Container, “and then put the relevant elements of the type Box inside the container.

neos-usability-tests-by-creativestyle-cms-opensource-1

 

At this stage, the testers had problems with unrefreshed nodes structure. It was really confusing and tutorials for editors have to pay attention to necessity of refreshing the node tree.

We also choose the same icon for one of the tabs as icon for Shortcut button what suggested the same functionality. It was confusing for some users so different details have to been taken into account while building the backend.

The testers often forgot about tabs or did not clicked on apply or publish button or they did not use live shortcut button and did not refresh live site. The changes were then unpublished and editors were a bit confused.

We should also consult with our clients ways of naming different elements on the site. There are cases when elements may be very similar in appearance and structure, and they can be coded as the same element. However, customer way of thinking can be more in terms of elements function. Sometimes it can be more convenient for a client to duplicate an element and give it other names and then group it in the appropriate section of the Create New panel.

neos-usability-tests-by-creativestyle-cms-opensource-2

 

Task 5: Add a new subpage.

Create a new subpage “Team” positioned between Adoption and Game apps in the menu.

The task was to add a new subpage. Then editors were asked also to insert nodes with descriptions of team members into the new page. Task 5 was about adding new page and content into it. It was quite easy for users and they usually did not have bigger problems.

Task 6: Translate the text.

In the task 6 users were asked to start the translation into Polish. The testers seemed to understand the concept of the translation, but the problem was to find the language drop-down menu, which disappeared when they selected an item other than the site.

Pros and cons of using Neos according to our testers
After the tests, our editors were asked to complete a survey to give us some feedback on things they liked and elements they had problems with.

What do they like? Here are their answers:

  • Clarity
  • Preview of the actual view
  • Fast editing / changing elements
  • Lots of features
  • Nice design (twice)
  • Fairly high level of intuitiveness
  • All elements are clickable
  • Easy transition to the live version after introducing changes and publishing
  • The colors (clear buttons on a quiet background)
  • Clear structure
  • A large number of options
  • Relatively intuitive
  • The interface has a very modern design that is easy to understand.
  • Colors and interface elements (icons, animated transitions) are very refined.
  • The display of the structure makes it easy to control the work.
  • Quite a lot of possibilities, intuitiveness
  • Clear
  • Intuitive, although at first glance you does not see everything that you can do
  • Transparent backend, not discouraging for beginners
  • There is quite a lot of elements similar to other applications

After the positive impressions it is time to talk about disadvantages. Some of the problems have been discussed in the article and can be associated with minimal information/training that were available to our editors. Some of the problems can be avoided by offering customers a good tutorial and a more complex initial training – for example in the video format or tutorials with screenshots.  As some of our editors told us that they were bored with instructions with screens and they actually never read them, short videos explaining basic interface  (like https://youtu.be/H5UdiFJvI7o) can be considered.

And finally we are expecting some fixings in the new versions of Neos.

Disadvantages of Neos (according to our testers):

  • Undo is not easy / is not entirely clear how many recent changes will be withdrawn
  • You need a lot of clicking to make changes
  • The link icon is identical to the icon to go to the live version
  • I did not see where I could add a link
  • Minor errors (one change lost)
  • On the low-resolution screen inspectors take a lot of space, so the work area is very small and requires much scrolling what difficult my work.
  • At the beginning it was hard to understand backend dependencies, but after some time spent with CMS you could get used to it.
  • ‘Add new page’ icon with an arrow is misleading.
  • Right inspector still does not seems to be intuitive for me (relations between tabs).
  • A little slow, not suitable for each resolution (1366×768)
  • Copying using the mouse and drag & drop function would be comfortable
  • I did not find how to undo small changes
  • It is impossible to do some changes on “home screen” – it would be better if some features are available right from the content canvas, not from the bar.
  • Structure tree was not refreshed after changes – there is a gap in “parallelism” of information / changes in the window
  • Width of the first panel (left navigate inspector) should be changeable

We should take a closer look at one of the above complaints – problems with responsiveness on the Neos backend. The testers had problems with displaying the full page view while working in the backend. Design of the site had responsiveness, but above 1200px screens it had the fixed width for the content container. It’s our mistake that shows the need for good tests on a backend for the most commonly used desktop resolutions (see chart below).

The short summary of the article will be another advantage mentioned by one of our editors: Actually it is quite cool to work on Neos!

pluswerk