HERE are two quotes from two of the best non-fiction writers and journalists in English of the past few decades.

First, Janet Malcolm from her book The Journalist and the Murderer:

“Every Journalist who is not too stupid or too full of himself to notice what is going on knows that what he does is indefensible…He is a kind of confidence man preying on people’s vanity, ignorance or loneliness, gaining their trust and betraying them without remorse.’’

In the book, Malcolm examines the relationship between the writer Joe McGinniss and the convicted murderer Jeffrey McDonald, who had consistently proclaimed his innocence of the murder of his wife and two children.

McGinniss, convinced that McDonald was innocent, contacted McDonald’s lawyers and told them he wanted to write a book about the case that would tell McDonald’s story. As a result, McGinniss was given access to lawyers, family members, friends and McDonald himself, all on the basis that he would write McDonald’s story. McDonald and McGinniss became close. 

But in the course of his research, McGinniss changed his mind about McDonald’s innocence. He did not tell McDonald this, but continued on with the relationship as if nothing had changed. 

When the book was published, McDonald of course felt betrayed. He sued McGinniss for breach of contract and fraud. At the trial, several journalists testified for McGinniss, arguing that McGinniss had misled McDonald in the public interest — for the story, I presume that means — and that this was ethically sound. The trial resulted in a hung jury. McGinniss later paid McDonald $300,000 without conceding fault.

Janet Malcolm argued that McGinniss had committed a fraud and had betrayed McDonald. She went further: she argued that all journalism involved betrayal of this kind. The interests of the journalist and the interests of the subject of the journalism are different and inevitably in conflict. And the more experienced and skillful the journalist, the more likely it is that this reality will be hidden from the subject — the “victim”.

Now here’s what Joan Didion said in her intro to Slouching Towards Bethlehem, her collection of essays and reportage published in the late 1960s:

“My only advantage as a reporter is that I am so physically small, so temperamentally unobtrusive and so neurotically inarticulate that people tend to forget that my presence runs counter to their best interests. And it always does. That is the last thing to remember; writers are always selling somebody out.’ ”

Those of you who have read Slouching Towards Bethlehem will know what Didion means. She is such an acute observer, so good at making herself invisible, in the sense that people forget that she’s a journalist and they say and do things they might not do if she was more intrusive, asked more questions, pushed them to reveal things. This is a different form of betrayal than the betrayal Janet Malcolm examined in The Journalist and the Murderer but it nevertheless does involve implicit — if not explicit –deception. It involves behaving in such a way that people you are going to write about actually forget what you, the writer, are doing.

Joan Didion and Janet Malcolm are saying similar things. One of the questions they raise is: who owns the story, the subject of the story or the writer? That’s only one aspect of what they are on about but it’s an important one.

In some ways, the betrayal in The Journalist and the Murderer is straightforward: the writer fooled his subject, in essence, lied to him. This seems to be a not very complicated ethical issue for writers and journalists. It is unethical surely to lie. It is unethical in any occupation and it is unethical in journalism as well. Even journalists who argue that their lying and misleading people is in the public interest are on shaky ground in my view. I know even this is contested – witness the journalists who testified for McGinniss – but I believe lying, except in the most extraordinary circumstances, is unethical.

But that’s not all Malcolm and Didion are saying.

Malcolm is arguing that being a journalist inevitably involves deception, that the ‘best’ journalists manipulate people in order to get a story, in order to get them to reveal things about themselves. They pretend intimacy, they flatter. At times they pretend to be interested in every word their subjects utter, no matter that they find the subject as boring as hell at times. 

If it will help, they tell people how interesting they are, and how important it is to tell their stories. They use silence so that people feel uncomfortable and then say things that they might regret, just to fill the silence. They hang around making themselves as invisible as possible in order to get people to forget that they are in the presence of a journalist. 

All this has a purpose that the writer cannot reveal to the subject. The writer is searching for his or her story: the writer’s story, not the subject’s story. 

This is why almost invariably, people feel they have been let down by journalists, that their story has been distorted, not really told. And it hasn’t been told. The journalist has told his story, a story that didn’t exist before he wrote it. There is inevitably a form of obfuscation at best involved in the relationship between the journalist and his or her subject.

Didion argues that her skill as a journalist was to make people forget she was a writer: indeed, to make herself as invisible as possible so that people would reveal things that they otherwise would never reveal and certainly not to a journalist.

This ability to get people to co-operate and, without knowing it, give up their story to the journalist, is fundamental to the journalism craft. The best writers, the best journalists, know this to be true and know that this can create ethical minefields.

Now, I am not putting myself in the same class of writer as Malcolm or Didion, but let me just illustrate this from my journalism, a story that I remember vividly because even now, decades later, I sometimes wonder whether I made the right decision — whether I was ethical, whether I betrayed the people who offered me their stories.

In 1983, I approached the Chief Commissioner of Police and asked whether I could spend a few weeks with the police in the inner Melbourne suburb of Fitzroy.

How many people get to have someone who listens to everything they have to say, with total focus on them, someone who does not interrupt, is not impatient with them? It’s the great gift journalists offer subjects and it’s a gift that invariably involves betrayal.

I told the commissioner, Mick Miller, that I would, as accurately as possible, convey to readers of The Age, the challenges and pressures facing police in the inner city. I told him that I wanted unrestricted access to all the police and all the goings on at Fitzroy and that no, I would not submit my stories to him for approval before they were published.

Different times! For a start, the commissioner took my call. Then he listened carefully to my proposal. I wanted to spend three weeks at Fitzroy and I wanted to be able to come and go as I pleased.  

He agreed!

Not only that, he took me to the station, lined up all the police, and told them I had his complete support. Different times indeed: the paper actually gave me three weeks at the station and a week to write the series. Imagine that!

Now when I spoke to Miller about my plans and about what sort of story I wanted to write, I was more or less telling the truth.

But I was not telling the whole truth.

Fitzroy back then had the biggest Aboriginal community in Melbourne. There had been tension between the Aboriginal community and the police and claims of systematic racism. This was a major reason for me choosing Fitzroy.

I wanted to explore the relationship between the Aboriginal community and the police but from the perspective of working police officers.

Was it okay that I did not tell Miller this? I assumed, I think, that there was no need, that he was astute enough to know why I had chosen the Fitzroy Police. Still, I was not entirely honest about what I was doing, what I hoped would be my story.

And certainly to the police at Fitzroy, I said nothing to them and, indeed, during the three weeks I was there, day and night, I never once asked a direct question about their feelings about the local Aboriginal community.

Joan Didion would have been proud of me. Or rather, she would have recognised a fellow practitioner in the art of unobtrusiveness. I did not take notes during those three weeks. I took notes later, at home. At the beginning, I sat silently in the station, in the back of police cars, at the pub where the police went for a drink. At first, they were wary, even pissed that Mick Miller had foisted me on them. Gradually, however, they forgot about me. I was just there. At that point, I started to ask questions, trivial questions that amounted to chit-chat, but all the questions were designed to show how interested I was in them and their lives.

How many people get to have someone who listens to everything they have to say, with total focus on them, someone who does not interrupt, is not impatient with them? It’s the great gift journalists offer subjects and it’s a gift that invariably involves betrayal.

… the fact is that my concern was only with the story. I wanted to write a good story, reveal something, give readers of The Age a compelling story. The story was in and of itself, my only concern.

Eventually, the police forgot that I was a journalist. I knew this when they started to talk and act in ways that they would talk and act only amongst themselves.  And yes, they said some awful, racist things about the Aboriginal community and this spilled over into how they approached Aboriginal people. I did not call them out for any of this, though I was shocked and, frankly, thrilled too, for I had my story.

When the series was published, the police at Fitzroy, many of whom had shared with me their frustrations, their hopes, their prejudices, were outraged.

They were furious too with the Chief Commissioner, though Miller never complained. Indeed, as a result of my stories, the Victoria Police appointed an Aboriginal liason officer whose job it was to build bridges, repair the relationship between the police and the Aboriginal community. I had betrayed them.

So I could argue that my stories were in the public interest and good had come from this betrayal. But the fact is that my concern was only with the story. I wanted to write a good story, reveal something, give readers of The Age a compelling story. The story was in and of itself, my only concern. And I believe this is the main concern of every journalist, every non-fiction writer.

This issue of betrayal matters whether the story is in the public interest or not.

In 2002, Don Watson, Paul Keating’s speech writer published Recollections of a Bleeding Heart, an account of his time with Keating and an account of the Keating  prime ministership.

It’s a wonderful book, brilliantly written and enthralling. Watson and Keating were close and I assume spent a lot of time reflecting on their period together in government.

There’s no point in writing if you are going to avoid offending, even hurting people. You have to believe in the story.

Watson’s book is not in any way an “official” version of that time. Keating did not approve its contents. He knew Watson was writing a book, but no more than that.

Now, when Keating read the book just prior to its launch, he was outraged. Though on any objective reading, it was a positive portrait of Keating as prime minister, Keating felt like he had been betrayed. He felt like it was a distortion. He felt belittled.

The falling out between Watson and Keating was bitter and public. The two have had no contact for a decade now.

Keating felt like Watson had stolen his story. And made it his own. Of course, it was Watson’s story. Watson has said as much. It was his take on that period of history.

How could it be otherwise?

In the end, the sort of betrayal discussed here is inevitable. There’s no point in writing if you don’t believe that the story is important in itself. There’s no point in writing if you are going to avoid offending, even hurting people. You have to believe in the story. Great non-fiction writers, great journalists — even not so great ones — believe in the importance of the story. The story is its own justification. And the story is their story. The writer owns it.

* Adapted from a lecture delivered at Melbourne University’s Centre for Advancing Journalism in February. You can listen to the full address here.