"The project has develop new methodology to support model-based combinatorial optimization. This work consists of three four main activities:
1. Mathematical definition of design problems and their efficient solution by known means in combinatorial optimization. In 2015-2017, we have defined several common UI problems not defined before, in particular in menu optimization, functionality selection, keyboard design, biomechanical design, and functionality selection. Some problems are defined at the level of a decision problem, which then can be solved using black-box optimizers. Others are defined using integer programming and permit the computation of guarantees for solution quality. We have also defined and presented first real-time optimizers for web page sketches and started to work on deep learning based optimization of web page layouts.
2. Interactive optimization in design. In 2015-2017, we have defined new methods that base on real-time optimization, robust optimization, and machine learning. They allow supporting the designer while avoiding overloading and providing sufficient means for steering optimization. COMPUTED has developed the first zero-effort approach to interactive optimization. This means that the designer's goal is inferred automatically and used to generate a diverse set of ideas on on alternative designs. Presently, we are working on the problem of how to infer regions of a graphical UI from its image only. This will help generalizing the use of these approaches beyond sketching to, for example, web pages etc.
3. Modeling of interactive behavior: We have advanced modeling of interactive behavior, in particular in the area of computational rationality. In this approach, we defined bounds/capacity limitations of humans and estimate the best achievable performance with a candidate design using Q-learning or other reinforcement learning approaches. This has been thus far applied in menus, information search, and visual layouts. One fundamental problem the project has looked at is how to infer models of users from realistic data. To this end, we have worked on inverse modeling of computational rationality models. The idea is to use ABC (approximate bayesian computation) to infer model parameters from naturalistic user data, such as clickstreams. We showed that parameters of the human sensorimotor systems can be obtained like this in menu interaction. This will enable an optimizer to learn models of users without running experiments. These models can then be used to optimize or adapt the UI for that user group.
4. Demonstrators: Using results from #1-#3, we have shown new results in challenging UI design problems from keyboards to menus and over to functionality selection, gestural input, web layouts, and biomechanics.
"
