You need a new pair of shoes.

En route to the shopping centre, you pull into a service station. Filling up the fuel tank, you distractedly watch ads on a screen on the bowser. Meanwhile the intelligence system at work behind the screen gets a fix on you. It catalogues you by age and gender. Are you wearing eyeglasses; a beard? Having sorted you into one of 18 demographic profiles, it serves up a targeted ad, and monitors your mood as it plays. This information is relayed back to advertisers so they can fine-tune their pitch.

The purveyors of this “digital-outdoor audience in real time” analysis – or DART – boast that it is “anonymously tracking over 2 million Australians weekly”.

Arriving at the shopping centre car park, your number plate is scanned. You walk into the mall, passing a discreet sign: “To improve your experience, we provide free WiFi and use WiFi monitoring in this centre. Please adjust your device settings to opt out of this service.”  The phone in your pocket leaves a trail of data breadcrumbs – where you linger, stop, speed up.

You pause near an advertising screen. Again, it’s watching you, rapidly curating the flow of content to match your profile.

You go to the shoe store and select a pair. Along with your credit card, you hand over a loyalty card in return for a modest bonus sometime down the track.

You’ve got your new shoes, and in the process delivered a motherlode of information which is soon being swapped and shared between retailing networks.

This scenario isn’t one that might play out sometime in the future. It’s entirely feasible that it is already happening at a shopping centre near you. The technology is in place.

The same facial recognition and video surveillance technologies used in airports and other high-security zones are being turned onto consumers by retailers – and not just to nab shoplifters.

They are part of an expanding array of systems keeping close tabs on Australian shoppers, monitoring purchasing habits, tracking mobile phone locations, accessing web browsing history, identifying an individual shopper’s demographic profile, even using emotion recognition software to read his or her mood.

Retailers can see how long a customer might linger in front of a display, even “what shelves you are looking at and for how long”: Dr Katina Michael, University of Wollongong.

It’s a consumer technology arms race, with retailers and businesses falling over themselves to plug into data which allows them to target consumers with finely-tuned advertising and marketing. The global retail biometric market is set to reach $1.6 billion in 2020, up from $625 million in 2015, according to the 2017 Deloitte Consumer Review.

Businesses argue the treasure trove of information they collect will improve the customer experience and streamline services. But these largely invisible and increasingly intimate intrusions into consumer behavior also raise serious concerns about privacy, and regulators are trailing way behind, according to legal, civil liberties, and privacy experts.

Surveillance systems are being combined with sophisticated analytics in such granular detail that they can see how long a customer might linger in front of a display, even “what shelves you are looking at and for how long”, explains Dr Katina Michael, an information technology and law expert at the University of Wollongong.

Right now, advertising company Val Morgan is rolling out its DART2.0 intelligent screens at service stations nationwide. Equipped with biometric analysis technology, these screens can read your face and features to determine your demographic profile and your mood.

The information the company collects and sells is anonymous, it states in its advertising. (Val Morgan did not respond to several requests from The Citizen for an interview.) But experts say that there is little comfort in this for consumers.

Given that the data captured includes your gender, age, ethnicity, location and the time of your visit, it’s no great leap to then figure out your identity, explains University of Melbourne expert in technology and cryptography, Dr Vanessa Teague.

“Scientists have re-identified ‘de-identified’ data from mobility traces, health data, search queries, movie ratings etc,” says Teague. “It’s been shown over and over again, in a wide variety of different domains, that de-identification of complex individual records doesn’t work.”

Then there are the ethical questions posed by the intimacy of surveillance. Last month, Westpac Bank revealed it was trialling artificial intelligence-powered video cameras to read the mood of staff, “so leadership can take the pulse of their teams across the organisation”.

We’re entering a reality of “uberveilliance”, says Michael. She defines the term – which she coined with a colleague in a 2009 academic paper, and which has since made it into the Macquarie dictionary – as “always-on, technology enabled, pervasive surveillance systems integrated into society, electronic devices, and even the human body”.

These technologies are designed to monitor and monetize people, she says, by tracking their “identity, location and condition – knowing what someone is thinking, why they do what they do, and how they feel when they do it. That is what these systems are trying to achieve”.

They reach into individuals in a new way, creating a profile that is best described as “behavioral biometrics”, Michael says. Their use in the retail sphere is “both subversive and subtle”.

Surveillance data is super-enriched when combined with the archive of customer information retailers routinely gather via the cash register on purchases by customers who have signed onto loyalty programs.

Some customers might be happy to trade off privacy in return for discounts and offers matched to their individual needs. But, as explained in the fine print, loyalty programs also trade information about their spending and lifestyles.

For example, when a shopper swipes a Woolworths Rewards card at the supermarket checkout, the information on what is in his or her trolley is shared with a range of partners, and can be combined with publicly available information and digital services – “including social media platforms”.

Retailers and other organisations are required to “de-identify” – that is, replace names with numbers – much of the data they collect and analyse to comply with the Australian Privacy Principles (APPs), which are enshrined within the Privacy Act 1988.

In reality, there is little regulatory oversight, says Dr Roger Clarke, an IT expert at the Australian National University and a privacy advocate. The regulator, the Office of the Australian Information Commissioner (OAIC), is weak, he says, and there are very few protections for Australian consumers.

“The law and regulatory frameworks have lagged behind technological developments and this presents serious privacy and also information security concerns”: Dr Monique Mann, Queensland University of Technology.

Dr Monique Mann, a specialist in privacy law at the Queensland University of Technology, says protocols and regulations capable of protecting personal privacy are trailing far behind the fast-evolving surveillance reality.

Mann says that existing opt-in and opt-out programs, where a shopper might agree to data sharing when joining a rewards program, are quickly becoming an “outmoded concept of consent”, questioning whether it is even possible to “opt out” in our increasingly connected society. The very act of entering a shopping centre implies you have agreed to the terms.

She says rules are urgently needed to ensure the data being captured is being used responsibly.  “The law and regulatory frameworks have lagged behind technological developments and this presents serious privacy and also information security concerns.”

In the US, privacy concerns recently found their way onto the public radar when mega-retailer Walmart filed a patent for technology that scans the faces of customers to determine their mood. The application revealed that it had re-purposed CCTV security systems to assess the mood of shoppers as they passed through the check out. Walmart hoped the system would identify unhappy shoppers, who could then be given special attention.

In a recent BBC forum on the commercial use of biometric data, a former head of the UK Border Force, Mr Tony Smith, warned that governments needed to legislate to head off its inappropriate use. “Private sector entities like stores having their own database of photographs” was, he said, an unexpected development. “It started as security, but is now being used for marketing … legislation needs to put down gateways where the data is being used inappropriately, and currently it isn’t doing that.”

The Australian Scene 

Australia’s two major shopping centre chains, Scentre Group (which includes Westfield malls) and Vicinity Centres (the DFO chain and Chadstone Shopping Centre), both collect movement data from customers.

Shoppers are notified on entry of the conditions they agree to when they step inside, and directed to a website for more detail. Some signs clearly state your mobile phone will be tracked unless you “change the settings on your device”, but provide no instructions about how to do that.

Vicinity’s privacy policy webpage states it may collect, among other things, your name, date of birth, address, details of your interests and shopping preferences, information about your visits to our shopping centres collected through the WiFi or mobile application, including “device identifiers, usage and location data”.

Westfield also collects personal information and its privacy policy states the centre tracks customers’ mobile phones. It says: “where devices are enabled to connect to, or are identifiable by, in-centre infrastructure (for example, in-centre WiFi networks or blue tooth transmitter (beacon) infrastructure), we and our third-party providers may automatically collect data from those devices including usage, type of device and location and proximity of your wireless device in-Centre, centre arrival and departure time”.

Both companies’ state they comply with the Australian Privacy Principles, that the data they collect about visitors is used for “management and security purposes”.

However, they provide little information on what data they do collect, how they collect it, how it is stored, who they share it with, or what they use it for.

The Citizen contacted management from both organisations for further information on their use of customer analytics, facial recognition technology and data collection.

Vicinity Centres declined to comment. However, in a statement to shareholders in August,  CEO Angus McNaughton said: “Our consumers now have access to free high-speed WiFi at our centres and the rich data we are gathering from this network is enabling us to gain insight into consumer behaviour, including dwell times, foot traffic and the way foot  traffic flows around our centres”.

Scentre Group (Westfield) provided details about the marketing benefits of the data it collects, but did not respond when asked how it collects this information.

► This story is co-published by Fairfax Media in The Sydney Morning Herald and The Sunday Age.