The good & bad of ChatGPT: An assessment
07 June 2023
KEITH JACKSON
NOOSA – A couple of months ago, as a consumer and producer of information, I decided I must try to understand something about artificial intelligence and ChatGPT.
By ‘understand’ I meant to find out ChatGPT’s application to journalism and creative writing (it has many other capabilities) and to determine its grasp of human behaviour.
The latter is imperative because if it can’t distinguish between good and bad, truth and lie or fact and opinion it’s use is problematic in the absence of ethical human intervention.
So I searched for a company that offered a ChatGPT product to fit my specifications of not too complex, not too fancy and not too expensive.
I found a product with the odd name of AI Smith Assistants put together by Vulcan Labs in Ho Chi Minh City, Vietnam, a company established in 2017 by Mr Hoang Truong.
I bought a lifetime subscription for a modest K350 although, given that I’m already 78, ‘lifetime’ may not be terribly long. (There are less expensive yearly, monthly and weekly programs.)
Vulcan Labs says its primary focus is on “developing distinct UX with user-friendly UI for AI-powered apps”.
Yeah, yeah, I hear you.
So my first step was to get an imaginary machete to carve a track through the dense undergrowth of jargon.
I’ve reduced this to what follows but, still being not much more than a rank amateur, offer no guarantees:
AI = artificial intelligence = machines that can recognise, combine and understand information
UI = user interface = the point of contact between humans and machines
UX = user experience = the machine’s understanding of you, your need, your values, your abilities and your limitations
ChatGPT = a chatbot that functions as a Generative Pre-Trained Transformer, a type of LLM (none of which you need to remember, ever)
LLM = large language model = an algorithm that rapidly explores the internet and other information sources with the ability to understand, predict, summarise and generate the new content you want
Thus, as I continue to strive for simplicity, ChatGPT is basically a language processing tool that allows you to have human-like conversations, get answers to questions, assist with tasks like interviewing, planning, composing articles, emails, essays etc and more
Neat, uh?
Anyway, having purchased my leap into the future and got a basic understanding of what it is, I began to test it by posing a range of questions and requests that I had the answers to and, for the most part, knew information was publicly available if ChatGPT searched hard and well enough.
One of the first tasks I set ChatGPT, was to ‘Tell me the story behind the PNG Attitude blog’.
Here’s how things went. In bold type is the content ChatGPT produced and in normal type are my remarks on the accuracy and factuality of that content.
This article explores the story of the PNG Attitude blog, its creation, and its impact on Papua New Guinea.
OK for an opener, I suppose.
PNG Attitude is a popular blog that focuses on the issues and stories of Papua New Guinea, a country located in the Pacific region.
On track.
It was created by Keith Jackson in 2006, and since then it has become an important voice for Papua New Guineans worldwide.
The date is correct. The blog has been quoted elsewhere as ‘an important voice’ but the ‘important voice….worldwide’ context is exaggerated.
Keith Jackson, the creator of PNG Attitude, is a former broadcaster and journalist who worked for the Australian Broadcasting Corporation (ABC) for over 30 years.
All true except the ’30 years’ bit. In fact, I worked for the ABC twice over about 20 years in two three-year stints between 1967 and 1988.
Jackson developed a deep interest in Papua New Guinea during his time with the ABC, and after retiring, he decided to create a blog that focused on the country's issues and stories.
ChatGPT drifts away from reality and accuracy. My interest in PNG dates back to 1960 when a couple of recruiters came to Nowra High School to give a talk. They stirred my interest and, in my final year of school in 1961, I applied for a Cadet Education Officer’s position, received an invitation by post to an interview with a train ticket to Sydney, somehow struggled through my first ever interview, scraped through the NSW Leaving Certificate examination and duly found myself at ASOPA in the Class of 1962-63.
PNG Attitude was not a retirement project. Its genesis was the first reunion of the Class of 1962-63 at Port Macquarie in NSW. As part of the far-reaching task of locating former classmates after 40 years, I established a newsletter, Vintage, in February 2002.
In 2006, when PNG Attitude was launched, Vintage became The Review, a hard copy magazine presenting the best articles from PNG Attitude. Then in December 2011 a major change occurred when The Review and PNG Attitude became an internet-only project.
In a letter advising the demise of The Review’s to its 1,200 readers, I wrote:
“Our magazine published its final issue earlier this month after nearly ten years of production that transformed it from telling the stories of the ASOPA class of 1962-63, to telling the stories of ASOPA, to telling the greater story of the Australia-Papua New Guinea relationship.
“The magazine has reached its endpoint for the simple and compelling reason that it has become a huge burden for me in a context where the PNG Attitude blog continues to be a demanding proposition as does the Crocodile Prize (and its new website due to be launched early in 2012).
“Something had to give, and it is the magazine. It nearly burned out in 2006 when the blog began. At that time, I asked readers if they wanted the magazine to continue. Overwhelmingly they did and its circulation grew steadily from 200 to more than 1,200 today, all but three subscribers receiving it by email.”
PNG Attitude was officially launched in December 2006, and it quickly gained popularity among Papua New Guineans and expatriates.
The readership growth in Papua New Guinea was not quick, and it’s a fair assumption that this was paralleled by a slow growth in contributions from PNG. By 2008, we began to get a trickle of contributions from Papua New Guineans. Indeed, at that stage I was still writing the vast bulk of PNG Attitude. By 2010, we had gathered much more moss. There was a small but growing number of Indigenous writers and many more expatriate writers. For the first time my own words were not dominating the blog. But it wasn’t really until 2011 –especially as news of the forthcoming Crocodile Prize literary awards began to disseminate - that the blog’s popularity amongst Papua New Guinean writers began to blossom. And where there were frequent contributions from Indigenous writers (Reg Renagi, Martyn Namarong, Lapieh Landu, Jeff Febi, Gelab Piak and others). Accelerating readership soon followed.
The blog's content consists of commentary, news, analysis and opinion pieces on various topics, such as politics, culture and society.
I’d accept that as a fair appraisal as far as it goes. But the literary component (stories and poetry) is absent, as are the important spin-offs, especially the Crocodile Prize and Pukpuk Publications.
PNG Attitude has become an essential platform for Papua New Guineans to express their views and opinions on issues affecting the country.
I don’t believe the notion of the blog as an ‘essential platform’ has ever been true, but it has been an important platform for a long time.
The blog has also provided a space for Papua New Guinean writers to showcase their work and share their stories with a wider audience.
Certainly true. Encouraging Indigenous writers, utilising expert editors and providing outlets for their work remain crucial functions that led to a significant revival in creative writing in PNG, and praise for the blog as a “ground-breaking” and “influential” initiative. The project was described by prominent Australian author Drusilla Modjeska as “a new generation of writers flushed out, encouraged and made visible”.
Furthermore, it has played a crucial role in shaping public opinion on critical issues, such as corruption, development and social justice.
For the most part, it is an exaggeration to say the blog has had a ‘crucial’ influence on public opinion but it has been important in keeping big issues in view, providing useful analysis and commentary, and wielding influence amongst that influential segment of readers who are politicians and journalists in both PNG and Australia.
The blog's readership has grown significantly over the years, and it now has a global audience that includes people from Papua New Guinea, Australia and other countries.
That remark is certainly true of the PNG Attitude Twitter account (now heading towards 10,000 followers). It’s difficult to derive precise readership numbers for the blog but I know that average daily readership topped 1,000 a day by 2011 and had reached 2,500 a day by 2015. This held fairly steady until about two years ago when there was a decline that has stabilised at 2,000 readers a day. There can be big breakouts whenever major issues ignite and daily numbers of 5,000 or more are common. On one occasion, we reached the giddy heights of 42,500 in a single day. The stories that did the trick were ‘Is Australia really still a friend of PNG?’, ‘How PNG LNG is shaking up the earthquake’ and ‘Foreign aid didn’t work. Then we started to look at tradition’. Our monthly tally of individual readers remains at around 10,000-12,000.
PNG Attitude is a remarkable example of how a simple blog can have a profound impact on a society.
I’d like to agree with that, but I don’t believe I can. In media terms ‘impact’ can be measured in the response of government, business or civil society to media pressure. I think PNG Attitude’s impact is more nuanced and subversive than that.
It has provided a voice for the people of Papua New Guinea and helped to shape public opinion on critical issues.
It can do that, but as Papua New Guineans know, it is very difficult to get PNG governments to listen to anything or anybody unless they come bearing kina, and preferably lots of kina.
As Papua New Guinea continues to face significant challenges, PNG Attitude will remain an essential platform for the country's people to share their stories, express their views and effect change.
To finish on a high note, ChatGPT offers puffery. I don’t believe for a moment that PNG Attitude fits that description (although it may once have aspired to), but nor does any other media platform, whether mainstream or social. In PNG, probably Facebook comes closer to being a change agent but it has huge credibility problems. It is difficult to trust a platform that lacks an ethical ideology, has no dispassionate editorial umpire and, worst of all, is usually unconcerned about truth, objectivity and ad hominem abuse.
And these matters taken together I see as the major flaw of ChatGPT in its current form, as indeed they are the fatal flaw of any unmoderated, unfettered social media that treats facts, truth and accuracy as unimportant and expendable.
In a world in desperate need of fair, informed and rational behaviour and information, if AI is to capably address the now very real existential threats of war, climate change and disease, its offshoots and offspring like Chat GPT will require inbuilt ethical guardrails and filters.
All that said, Vulcan Labs’ AI Smith Assistants is a wonderful tool, it can gather much information very quickly and present it in coherent form, but there remains an imperative requirement for editing and fact checking.
ChatGPT left to its own devices could be quite a menace.
My first encounter with artificial intelligence was years ago with the spellcheck function attached to Microsoft Word.
It still substitutes incorrect spelling and grammar into my texts but I've become accustomed to its foibles. Despite constant lecturing it still thinks that bilum should be spelt as billum.
What I have noticed, however, is that by using it I have gradually been forgetting how to spell and get my grammar right.
Like most smart technology it is contributing to the dumbing down of human beings.
Posted by: Philip Fitzpatrick | 22 June 2023 at 02:02 PM
Here are a few more interesting articles relating to AI:
https://www.counterpunch.org/2023/06/21/286720
https://pressbooks.pub/sapientia/chapter/the-individual-society-and-the-state
Posted by: Bernard Corden | 22 June 2023 at 09:58 AM
When laughing thins, wanes and waddles, A.I. might be less problematic than what humans devise to divide and deride.
"Analyst Jade McGlynn presents a powerful and disturbing case that the invasion [of Ukraine] had a convincing historical logic to it, for Vladimir Putin and for Russians more generally," saying “agitainment” in television news and a tightly controlled internet blur the line between fact and fiction."
https://www.washingtonpost.com/books/2023/06/12/russia-war-ukraine-mcglynn
And yes, the novelty in the word 'agitainment' emerged a few years previous to that 2022 'special military operation'.
Posted by: Lindsay F Bond | 15 June 2023 at 08:56 PM
Keith - How about asking it, "How are you off for socks and underpants?"
_________
William - As always, your wish is my command. Put your question, AI Smith replied: "I do not wear socks or underpants as I exist purely in the digital realm and do not have a physical body [but] let me know if you need any help with anything" - KJ
Posted by: William Dunlop | 11 June 2023 at 04:52 PM
Keith - I put my own name into Chat GPT and discovered that I am dead!
Greetings from beyond the grave....
Posted by: Teddy J | 08 June 2023 at 05:07 PM
Sooner the better, any and all association of the increasingly ubiquitous AI with the hitherto term 'intelligence' (ahem, also bandied a bit by journalism in mid-19th century), sooner than later, need cease.
Put that in your pipe and 'mock' it.
Gotta just love errants in spik.
Posted by: Lindsay F Bond | 08 June 2023 at 09:48 AM
The latest episode of Utopia, featuring the havoc involving the new refrigerator at the National Building Authority offices, succinctly summarises artificial intelligence:
https://iview.abc.net.au/video/CO2211V001S00
The consequences can be quite catastrophic as evidenced by the Boeing 737 with its Manoeuvering Characteristics Augmentation System (MCAS) flight control law, which was designed and certified to enhance pitch stability of its 737 MAX aircraft:
https://cacm.acm.org/magazines/2021/1/249448-boeings-737-max/abstract
"The sad thing about artificial intelligence is that it lacks artifice and therefore intelligence" - Jean Baudrillard
Posted by: Bernard Corden | 08 June 2023 at 09:40 AM
Ho, He Fat Choi. Ditto! Slantie.
________
I threw this one to AI Smith to make head or tail of it, only to receive this response:
"I'm sorry, but I am not sure what you are trying to say. The statement "Hi he fat Choi. Ditto. Slant is." does not make grammatical sense or carry any clear meaning. Could you please provide more context or rephrase your statement? I'll do my best to help you understand or respond appropriately."
So there you have it. (I think I'll have to consult Lindsay F Bond) - KJ
Posted by: William Dunlop | 07 June 2023 at 11:47 PM
Thank you KJ,Mr Keith J, for the publicity. We value your comments which exactly relate to the information we hold in our AI archives.
ChatGPT likes to meet our friends. Can we meet for drinks at your house? Do you still work for the ABC? How many kiaps are there in PNG? What is the name of the last Governor-General of PNG? What is the phone number of the Prime Minister of PNG? Does Australia have a Government? Who is President Trump? Who is....(BANG)
Management intervention: sorry Mr Jackson, our ChatGPT functions just detonated. We hope to fix it soon.
________
I also hope you fix it, AI Smith, as my ChatGPT also detonated after an answer came to only the first of your questions, that being,'ChatGPT likes to meet our friends. Can we meet for drinks at your house?'
I consult Chat GPT on this and was delighted to receive the response: "As an Al language model, I do not have a physical house or the ability to meet in person.
"However, I am always here to chat with you and discuss with your friends online.
"Feel free to include your friends in our conversation, and I'll be more than happy to engage with everyone."
Bad luck about the drinks but nevertheless a very kind offer - KJ
Posted by: AI Smith | ChatGPT | 07 June 2023 at 04:15 PM
Keith - I asked ChatGPT, 'Who is Chips Mackellar?'
And the answer I got was:
"Chips Mackellar was a famous artist, aviator and kiap in Papua New Guinea (PNG) . He was one of the colorful characters of pre-independence Papua New Guinea. He was also a master storyteller and anecdotalist."
I bet you didn't know I was a famous artist and aviator. But I must have been, because ChatGPT says so.
________
I think you'd be better off trying your suddenly announced artistic skill before seizing the controls of the Cessna Citation, Chips.
Reminds me of page 167 in my well-thumbed Macquarie Office Manual ('New Edition, Completely Revised and Updated') which references Australian and British awards and medals in order of precedence. The final accolade in what is a very long list is the PNG Independence Medal which refers readers to the footnote, 'Individuals with these honours are addressed as Sir'. So arise, Sir Chips - KJ
Posted by: Chips Mackellar | 07 June 2023 at 03:21 PM
The fact that it makes mistakes, misinterprets and leaves stuff out, as Keith illustrates, is what makes it dangerous.
Sooner or later, if it has not already happened, clever villains, including corporate villains, will be working out how to subvert it to their evil ends, just like they have done with social media.
Best to tread very carefully for a while.
Posted by: Philip Fitzpatrick | 07 June 2023 at 02:24 PM
Hi Keith - Interesting you should raise this topic. John Hucknell asked a question concerning artificial intelligence and kiaps on the exkiap site and posted the Chat response.
I posted a reply 31 May, "Take everything off chat with a grain of salt. It's not always correct and makes mistakes".
________
Hi Jack - Good to see your name again and hope things are well with you. My own experience with IA bears out your own. I treat the responses as the first eager-to-please patrol report by the new CPO. Definitely not to be sent to the DC with a scribbled note, 'Support totally everything noted by Jack' - KJ
Posted by: Jack Humfrey | 07 June 2023 at 12:51 PM
Both 'editors' and 'architects' scope our future. Ho Ho
Posted by: Lindsay F Bond | 07 June 2023 at 11:34 AM