System allows real-time fusion of data from human and machine sources filtered through a graphical user interface to enable target tracking in a complex environment.

Much progress has been made on developing the ability to track a target in an unobstructed area, however, target tracking in a complex environment having multiple obstructions poses challenges that have yet to be addressed. The system uses a graphical user interface (GUI) to fuse information from human sources (soft data) with information from machines (hard data) in a manner that autonomous agents or humans can use to more effectively track targets in a complex environment. The GUI makes use of existing touch-based hardware such as smart phones or tablets to develop an environment that can be as simple as a map or as detailed as a cityscape, based on the requirements of the user. User input, obtained through a touchscreen, is used to identify a target and a particle filter utilizing a Bayesian mathematical framework determines the probable location of the moving object. The system allows for real time communication and sharing of data between users that does not rely on a shared operational understanding nor upon any verbal communication. Furthermore, the GUI “landscape” is dynamic and adjusts in response to data received from users.

This system can be implemented on existing GUI hardware such as tablets and smartphones to achieve a variety of tasks requiring collaboration and data exchange between humans and machines. For example, the system and software could be used by agricultural operators to determine when crops should be harvested, treated for a disease or fertilized. The basic model of the system would be updated based on real-time information – such as weather conditions or insect populations – input by human users. The technology also has applicability for law enforcement, first responders, industrial inventory management, and medical professionals.