Learning Models of Handwriting for Structured Texture Synthesis

The proposed research aims to produce images of handwriting using a new generative mathematical model of different people's handwriting styles and the specified text strings to be "forged." Our model uses image-fragments of real handwriting samples and learns what fragments are compatible under what deformations.

Our intended analysis/synthesis applications include quantitative authentication of handwriting in legal cases, inpainting of destroyed sections of historical documents, imitation of handwriting for banking purposes, and eventually, writer-specific handwriting recognition and drawn-sketch interpretation.

The specific objectives of this project are to:
- Automatically synthesize handwriting of novel text but in a particular person's style.
- Measure the likelihood that a given sample was written by a specific individual.
- Build a system that suggests what examples of handwriting should be captured next.


The project scope is limited to just handwriting to assess feasibility. At its conclusion, we expect to further pursue the following additional objectives:
- Demonstrate the synthesis and analysis approaches on large-scale forgery studies. The goal is to measure our forgery and forgery-detection rates to help transfer these techniques into industrial practice. Abuse of the technology for forgery creation could be limited by filming an individual while writing, to confirm that the synthesis was not automatic.
- Build a similar adaptive authoring system to help synthesize cartoon animation. Replace strings of characters used in the present project with a structure based on limb-configurations (2D joint-angles) of a stick-figure.
- Explore a unified framework for example-based generative models. In many domains, better synthesis should be possible when multiple examples are available. Findings from this near-term research on handwriting should generalize, so that structured acquisition of training examples steadily improves the quality of synthesized drawings, 3D shapes, and video post-production.

This project will have impact on at least three communities: i) professional Forensic Document Examiners, ii) forensic scientists who study handwriting (both synthesis and recognition), and iii) vision and graphics researchers studying texture synthesis. With the UK's imminent dissolution of the Forensic Science Services (FSS) because of cutbacks, there is no substantial national hub for forensic science to be developed, and for the expertise to be disseminated to investigators and prosecutors. At least in the case of handwriting synthesis and analysis, some of that burden could be taken on by the PI and the host organization, as initiated in this project.

The impact of the proposed work will be far-reaching. Benefits include the ability to synthesize convincing imitations of different individuals' handwriting, and critically, the ability to numerically measure the probability that the handwriting on a questioned document was written by a known individual. Our intended analysis/synthesis applications include quantitative authentication of handwriting in legal cases, inpainting of destroyed sections of historical documents, imitation of handwriting for banking purposes, and in the longer term, writer-specific handwriting recognition, and eventually drawn-sketch interpretation.

To increase the likelihood of impact, to share expertise, and to raise the profile for the new approach of "structured texture synthesis", the PI and RA will organize a one-day workshop near the completion of the one-year project. Naturally, all research findings will be subjected to peer review and disseminated at conferences, in journals, and in specialized seminars for professional handwriting analysis / synthesis practitioners.