Project Jacquard

Weaving touch and gesture interactivity into any textile using standard, industrial looms.

Project Jacquard is  makes it possible to weave touch and gesture interactivity into any textile using standard, industrial looms. Everyday objects such as clothes and furniture can be transformed into interactive surfaces.This is possible thanks to new conductive yarns, created in collaboration with our industrial partners. Jacquard yarn structures combine thin, metallic alloys with natural and synthetic yarns like cotton, polyester, or silk, making the yarn strong enough to be woven on any industrial loom. Jacquard yarns are indistinguishable from the traditional yarns that are used to produce fabrics today.

Jacquard presents a unique material technology that is comprised of both hardware and software innovations. On the hardware side it starts by seamlessly integrating sensing capabilities in textile. It starts with the material science research to be able to embed conductive threads in normal materials like wool, cotton, denim, silk – to a level that it can withstand industrial processes textile through. It also demonstrates the technical engineering feat of developing hardware processes to connect complex grids of conductive thread into microelectronics.

On the software side software platform implements firmware for filtering and processing data directly from textile to advanced algorithms for calibration, detection and allowing different aspects of multi touch sensing to happen – profile, movement, direction, pressure, momentum. Down to implementing the protocols for communication between the textile and computing devices.

Research & Development

Experimenting with textile & services

The starting point for project Jacquard was to experiment with the data outputted by the textile grid. Initial prototypes included a data pipeline from textile to an Android phone through a PCB board that read capacitive oscillation on x,y axis of the textile and transmitted the raw data to an Android App. On the application we developed a system middleware layer that interpreted the raw data and abstracted the data into signature gestures (swipes, holds, touch). At that point we created the first prototypes using gestures to activate other phone functions such as placing a call to another phone.


Functional prototype: swipe gesture to place a call (with a predefined number) to a phone.

Defining Use Cases

As a second step we developed a series of user experience sketches demonstrating different potential applications of Jacquard in real world scenarios. These scenarios showed the different possibilites for gestures, integrations with other technologies and use cases to substantiate the use of textile as an input interface for a variety of uses.

These scenarios were then used to create the blueprint for a textile-specific gestural language and a complex system platform that could support all potential integrations. In the scenarios we experimented with possible integrations (from textile to service), input (touch, swipes) and output (minimal feedback using flexible LED arrays).


Interaction sketch: requesting a cab with one tough, feedback through flexible LED display.

Creating a gestural language

We created a new textile gestural language based on the textile form not being flat and ergonomic feature for the textile being worn. The gestural language included three main groups of gestures: (1) intetional gestures (2) infered gestures (3) semantic gestures. The intentional gestural language is based on intentional interaction between user and garment by expressing simple action with a finger or hand with the Jacquard mesh. Infered gestures that the garment could detect different user actions (such as waving, running, walking) combining data from the Jacquard capacitive mesh as it’s flexed with additional sensors (gyroscope and accelerometer). In the third group of gestures users could customize the device to recognize simple semantic symbols that would be then associated with actions.

Visualising Capacitive Data

To better understand the capabilities of the sensing mesh we visualized the changed of capacitance across the Jacquard mesh. In that way we could better understand the different possibilities of interaction and potential applications. Taking the visualizations we defined different directions for the interactions and explored the different properties that are unique to the Jacquard mesh such as detecting pressure of pressing, multi touch interactions, whole hand interactions and 3D touch (such as wraping the fingers around the arm).


Visualizing capacitive properties of the Jacquard mesh.

Developing the Jacquard Platform

The Jacquard platform extends beyond the interactivity with the textile into an ecosystem of connected clothing. The platform includes several components that range from the Jacquard capacitive mesh itself, to the electronic stack that interprets the raw data into higher level gestures. It expands into a mobile client app that provides user the level of customization and control of the mesh. The app is then part of a larger ecosystem of microservices managed by a central server that provides the end to end integration from the textile mesh to input/output actions with other apps or third party services.

Launch at Google I/O 2015

Project Jacquard was launched at Google I/0 2015 with a live demo as part of the ATAP session and an installation that let conference attendees to experience different demoes of the platform that could be experienced in the festival main hall.

My Role

From the start of Jacquard I was lead creative technologist and interaction designer for the project. I’ve worked closely with project technical leads Ivan Poupyrev and Carsten Schwesig (Google ATAP) to develop the vision for the platform, I’ve created the initial prototypes and technical integrations with computing devices, led the core interaction design and development of the gestural language for touch sensitive fabric.

Awards

Cannes Grand Prix – Product Design, Impact Innovation (2016)
D&AD Wood Pencil – Product Design, Innovative Product Design (2016)
D&AD Graphite Pencil – Digital Design, Tech Innovation (2016)

Further Reference

Project Website
Watch Launch Film
Developing with Project BloksJacquard Launch Keynote at Google I/O

Awards

Cannes Grand Prix – Product Design, Impact Innovation (2016)
D&AD Wood Pencil – Product Design, Innovative Product Design (2016)
D&AD Graphite Pencil – Digital Design, Tech Innovation (2016)