The "networked BMW" of the future recognises drivers by their face

Research project: biometric personal identification

Munich. A critical element in the exceptional levels of comfort on board a premium vehicle is the degree of individualisation. Playing a leading role – alongside high-class looks and quality – in achieving this individualisation is the scope for personalisation of the ergonomic, control, seat and infotainment functions. Drivers of luxury models are already able to store these settings on their car key. Like a PIN number or password, a key can of course be passed on to other people. Biometric properties, on the other hand, cannot: physical features are unique. In order to find a way to guarantee independence from the key and thus also ensure protection of personal data, the BMW Group is conducting intensive research into the potential of video-based personal identification. This biometric authentication process builds on the existing concept of key personalisation, but identifies the driver unequivocally by distinguishing facial features.

Without the need for any active movements – as required for a fingerprint sensor, for example – an infrared camera integrated into the vehicle scans the driver's face as soon as he sits down in the driver's seat. The data recorded is then cross-referenced against the facial features stored in a database. Only when the driver has been clearly identified are the comfort settings in the BMW prototype (for the seat, mirrors, steering column and radio) automatically applied to the individual involved. In the future this capability could even extend to linking up personal data – such as the driver's personal telephone book or the navigation system's address book – and the car's immobiliser to the camera-based driver memory system.

Driver to vehicle: from the first meeting to greater familiarity Before biometric authorisation can take place, the system has to familiarise itself with the driver by analysing the structure of his or her facial features for the first time. The processing software interprets the infrared image and surveys the distinguishing areas of the person's face. These include the upper edges of the eye sockets, the area around the cheek bones and the sides of the mouth. The registration process sees a grid placed over the scanned image of the face in order to localise the distinguishing areas and record their position and distance from one another. These facial features are stored as a template in a database under a secret ID chosen by the driver. The comfort settings (seat, mirrors, steering column, radio programme) selected by the driver are then matched to the face and saved in the system's memory.

The system can store the details of any number of drivers. This means that, if approved by the driver, an unlimited list of people can be added to the database, complete with their biometric facial features and comfort settings. Every time they get into the vehicle, each person's template is then on hand to serve as a comparison model. In the current prototypes recognition is completed in around five seconds – in the same way as the identification process took place the first time the driver sat behind the wheel. Only now, the person's distinguishing facial features are compared with the entries in the database. For example, in vehicles which are used by several different people either privately or professionally, these biometric identification methods could in the future also allow personal information (e.g. a telephone book and navigation destinations) or even sensitive data, such as the PIN number for a SIM card, to be assigned clearly and securely to the driver concerned.

The biggest challenge: light
In contrast to the development of video-based personal identification systems for use inside buildings, the BMW Group development engineers have to contend with constantly changing light conditions – over which they have little control – in adapting the process for vehicles. For example, sun shining onto the vehicle from the side casts an intense shadow over the driver's face, making it tricky to identify. And when beaming in from the front, the sun can cause distortion and therefore dazzling of the camera. Minimising these external influences to allow the system to function correctly represents the biggest challenge in ensuring that video-based personal identification technology is sufficiently robust for use in series-produced vehicles.

An owner's manual unique to every vehicle

Basis: the web-based personalised owner's manual of BMW Group

Munich. All the main vehicle functions in BMW Group products are designed for quick and intuitive operation by the driver. There will always be some situations, however, where the driver has no choice but to resort to the owner's manual – for example in the case of less frequently used or unfamiliar functions. But increasingly, proliferating vehicle functions and options, along with ever more personalised products, are exposing the limitations of printed manuals. The BMW Group is working on a solution to this problem in the form of a personalised web-based owner's manual specially adapted and matched to the customer's individual vehicle. Notes like "only for versions with ..." or intricate tables where the owner has to match the data he wants to the relevant engine versions will then become a thing of the past. When the customer configures his vehicle at the order stage, the personalised owner's manual will be configured at the same time. That would be unthinkable with a printed manual.

Integrated information is always up-to-date

This personalised solution, along with further, related functions, can be provided via a personal, customer-specific web portal. Customers can view their vehicle in multimedia form on their home computer, for example with animations to help explain technical concepts. Another helpful feature is an image search function tailored to the customer's vehicle. The web-based application also allows users to search for retrofit equipment – even before taking delivery of their vehicle.

The operating instructions can be continuously updated throughout the life of the vehicle, as and when retrofit equipment is added or modifications are made to the vehicle during servicing. So if vehicle accessories like a ski bag or trailer coupling are added, or vehicle software is updated, the manual is immediately updated too. Then, whenever the customer visits the portal, the information he finds there will be the right information for his vehicle and always up-to-date.

Technical content of the personalised web portal

The content of the web-based personalised owner's manual is based on existing data – for example equipment descriptions – from existing production systems, collated with customer and vehicle-specific data taken from sales and service systems. The different data elements are logically associated and incorporated in data models. The HTML presentation is generated by the user's browser.

Ideas for the future
Alongside the web-based version of the personalised owner's manual, the BMW Group is also working on a vehicle-integrated version, which would offer scope for even greater personalisation. A further feature of this version is that it would be possible to access the personalised information quickly and at any time right there in the vehicle.

Plucking entertainment out of thin air: wireless integration solutions

Bluetooth audio streaming and wireless USB inside your car

Munich. With their wide range of integrated infotainment functions, the premium cars produced by the BMW Group have long since entered the realm of "networked vehicles". One of the major challenges here lies in how to link up the multimedia devices – which come onto the market in quick succession – to the vehicle in a user-friendly way. Having integrated a mobile device into the car, there needs to be a control system in place inside the passenger compartment which ensures that it can be operated intuitively, without causing distraction and in an ergonomically sound manner. The application itself and customer-specific data remain within the mobile device, while its operation and output are handled by the vehicle's HMI (Human Machine Interface), i.e. its operating logic. In addition to the convenience offered by the straightforward and direct usability of these devices inside the vehicle, the benefits to the customer include enhanced security and assistance in using the integrated technology in accordance with driving laws. In BMW Group vehicles these benefits are already possible for music playback – using an Apple iPod (or many other music players) through the USB audio interface – and for wireless hands-free mobile phone calls via Bluetooth, for example. The BMW Group is monitoring and looking into a variety of different data transfer technologies to enable the widest possible future utilisation of mobile devices offering attractive and useful applications without the need for potentially awkward wiring in the car.

High bandwidth for sound and image: wireless USB
Wireless USB is the cable-free variant of the widely familiar USB (Universal Serial Bus) standardised interface. The new technology not only renders a data transfer cable superfluous, it also allows the use of applications requiring high bandwidth, such as video transfer. Wireless USB is based on ultra-wideband (UWB) short-range radio technology. UWB is a technology which could enable future IT and entertainment electronics products (e.g. laptops, mobile phones, digital cameras and televisions) to transfer large quantities of data over short distances wirelessly. Technology currently allows for transfer speeds of up to 480 Mbit/s. That equates to the bandwidth of the wired USB 2.0 standard. As well as the convenience of wireless connectivity, wireless USB also ensures data security through 128-bit AES encryption.

Music out of thin air: Bluetooth audio streaming
The BMW Group is currently working on an in-car Bluetooth-based playback solution for music files. In the future we can look forward to stereo-quality music streaming from a compatible device via Bluetooth to the vehicle's infotainment system, without the music files having to be transferred to the vehicle before listening. This capability will use the Bluetooth A2DP (Advanced Audio Distribution Profile) and AVRCP (Audio/Video Remote Control Profile) standards. A2DP takes care of the actual audio streaming, while AVRCP allows the audio output to be controlled through the vehicle's infotainment system. Devices are currently available based on AVRCP 1.0 and 1.3, which allow audio streaming but only with simple remote control functions such as start, stop, pause and skip. Bluetooth audio streaming will only be able to offer the convenience you expect in a BMW Group car – i.e. functions such as track selection – when AVRCP 1.4 arrives. The development of this standard is ongoing.

A key factor in the decision to implement this kind of technology in vehicles is the benefits it offers to customers. The nature of in-car products and their functionality are therefore dependent on the range of mobile devices produced by manufacturers and the implementation of the requisite interfaces. BMW Group experts closely monitor market developments to ensure that the company offers solutions in its cars that will allow the use of attractive new applications in mobile devices. Added to which, the CE4A (Consumer Electronics for Automotive) working group of German car manufacturers has been set up to represent the automotive perspective in the drive for standardisation.

Open to new technologies: the dynamic HMI generation

Development project at BMW Car IT GmbH integrates plug-in services into existing user interfaces

Munich. Electronics are behind virtually all automotive innovations today. In particular, infotainment and assistance systems are seeing steady growth: radio, navigation, telephone, voice dialogue, TV and MP3 systems, and even parking assistants and lane departure warning systems, have been a familiar feature in vehicles for some time now. On BMW's premium vehicles, all systems that allow user input are operated via a typical "automotive HMI" (Human Machine Interface). This vehicle-optimised control and operation system will in future allow infotainment services to be integrated into the interface which have not even been developed at the time the vehicle is delivered to the customer. This will make it possible to rectify the growing discrepancy between long vehicle lifecycles and the very short innovation cycles of consumer electronics applications. Model-based HMI development in the BMW Group takes into account this dynamic aspect by opening up the vehicle for "dynamic plug-in services" in the form of new mobile devices, online applications or software downloads. In the future, the customer will be able to operate these services quickly, simply and in individually adapted (vehicle-specific) form. In BMW Group products, this "vehicle-specific" aspect means that the services can be used intuitively and without causing distraction, and that they are ergonomically optimised. Benefits for the customer include not only the convenience factor of being able to use new devices as they come onto the market but also enhanced safety and legal compliance.

Flexibility through semantic information
Dynamic configurability of the various control functions and information displays is achieved by integrating into the plug-in services a semantic description of the relevant user interaction. Typical automotive graphic user interface elements can then be created using generating rules. Use of "semantic web" concepts ensures clear terminology for the HMI description of the service and its operating requirements. The HMI description of the service provides detailed information about the type of service and about control and display aspects. It is not necessary for the author to know in what concrete user interface in what vehicle model (and thus on what head unit) this information will be implemented. For example, the term "audio player service" can be used to identify any music player, whereupon the HMI will generate control buttons for the relevant user operations such as "play", "stop" and "pause". The description of the user interaction only needs to be generated once and can then be used for any vehicle model.

Another advantage of semantic information is that it allows interaction between services which have had no previous knowledge of each other. All they need to know is what type of service they can interact with. For example, if a location-based search service identifies itself as a provider of geographical addresses, the user will then be able to transfer the search results, i.e. address details, straight to the navigation system as a destination address at the press of a button. If the service also provides telephone numbers, these are automatically transferred to the telephone service. In this way the dynamic HMI allows the user to combine any number of services in a wide variety of ways, even if the services in question did not even exist at the time the vehicle was built.

1) Image 1: Research project Driver Memory
2) Image 2: Project Bluetooth Audio Streaming
3) Image 3: Research project Dynamic HMI
4) Image 4: Project Automotive open infotainment-platform