ANALYSIS & FINDINGS

This section projects the narrative of our project with extensive analyses of our data using various visualizations for each part of the narrative. Here, we analyze the general sentiments around ChatGPT, sentiments from different industries such as Technology, Marketing, and Education, and the sentiments around ChatGPT in relation to employment rates. Freely navigate through different sections of our Analysis & Findings page using the navigation buttons below.

General Sentiments

In this section, we explore the general sentiments around ChatGPT during the first month of its launch, conducting sentiment analysis using tools such as Voyant Tools and Vader Script in R, and reporting our findings using charts created with Tableau.

This bar chart shows the number of tweets around the topic of ChatGPT over the span of the first month after Open AI released ChatGPT. Through this visualization, we can observe that the number of tweets about ChatGPT dramatically increases during the first couple weeks of launch, peaking at over 16,000 ChatGPT tweets just in a day around December 8th, 2022. After this stark increase, however, the conversation seems to slowly decrease day by day, but not to the extent where it completely diminishes.

Conversation about ChatGPT is still ongoing each day with an average of 4,000 tweets per day, and individuals are consistently talking about this new technology that is yet too “good”, “amazing”, and “new” (sentiments found through Voyant Tools word map below) to the world, intriguing individuals to this day with its advanced functions, usability, problem-solving capabilities, and the influences it’s having in the world.

Based on these results, we will further explore how many of these tweets, on average, are associated with positive sentiments and similarly with negative sentiments, and extracting the ratio; in doing so, we aim to delve deeper into the public’s impression of ChatGPT around the first month of launch as well as what downsides and concerns about ChatGPT simultaneously arose during this time, if any. 

Word cloud generated from all tweets in the dataset

The visualization above represents some sentiments around ChatGPT. The visualization was created using Voyant Tools and unnecessary terms were manually entered into the stop-words list to generate the sentiments. Through this visualization, we aimed to explore the general sentiments around ChatGPT on Twitter, and whether the sentiments were mostly positive or negative. 

At once, we are able to observe that there are positive words such as “successful”, “useful”, and “inspire” nearly dominating the word map; there aren’t many negative sentiments except for a few neutral words that do not necessarily carry strong sentiments (“text”, “got”, “code”, etc.). This indicates that the overall impression of ChatGPT among the public is mostly positive, being characterized as a leading productive tool that could help users achieve their goals.

As this AI chatbot was just released and thereby still fresh to the public and various industries, individuals seem to be exploring novel functions and finding them impressive, which could have contributed to the overall positive sentiment around ChatGPT; in other words, not many problems with ChatGPT would have been found yet, given that the tool was recently developed. ChatGPT’s ability to write essays with given prompts, generate codes, and even generate different responses for each user could only amaze the public.

Hindered by these impressive functionalities, however, are the concerns that arise with these fast-developing artificial intelligence technologies; AI technologies continue to consume information and data from individuals to create the best models that could predict and even mimic human behaviors.

Though users could be benefiting from this technology at the moment, problems such as ChatGPT stealing existing works, generation of bias within the program, and inaccuracy of the results could and have been emerging.

The visualization above helps us analyze the compound sentiment scores for each tweet that was categorized as positive, and sorts them from the tweets with highest compound score to the lowest.

The three tweets that were identified as the most positive and that were actually related to ChatGPT were the following:

• “Dear friends and colleagues! I wish you a wonderful festive time, health, happiness, and professional success! To celebrate with #AI, I asked #chatGPT to write me a romantic Xmas poem, and it was lovely”

• “Usually I handpick an amazing #C64 winter artwork, but for this season’s holiday greetings, I asked #ChatGPT to create a prompt for #StableDiffusion – after all, 2022 was the year of amazing ML model developments! Happy holidays and a good start into the New Year 2023 to y’all!🤓 ”

• “I know, ChatGPT was Tuesday’s topic. But I’m still playing with it. And it turns out it’s amusingly good at role-playing. I came up with situations both practical and fantastical and assigned us characters to play. It did a great job and it was pretty fun. Here are a few samples.”

The tweets above seem to be praising the work done by ChatGPT such as having it generate a poem, create a prompt, and even provide a character in role-playing.

In contrast, the visualization above helps us analyze the compound sentiment scores for each tweet that was categorized as negative, and sorts them from the tweets with highest negative compound score to the lowest.

The three tweets that were identified as the most negative and that were actually related to ChatGPT (and do not have excessive profanity) were the following:

• “This probably marks me as someone who just doesn’t “get it”, but I’m having a really hard time getting worked up about stories where someone asks ChatGPT/whatever to do something racist or unethical, and then the AI produces something racist or unethical…”

• “If you think ChatGPT is going to kill off your job, just remember that: Calculators were supposed to kill off math. Fitness machines were supposed to kill off outdoor recreation. TVs were supposed to kill off the theater. No one can predict the future”

• “ChatGPT has a proven bias against Christianity. When asked to write an essay detailing the evils of Christianity, it “reluctantly” complies and lists off slavery, crusades, and “”homophobia””. Asked to list off evils of judaism or islam, it immediately babysits them. PURE EVIL!”

These tweets seem to express some concerns over the confusion with the idea of ChatGPT or other artificial technologies, uncertainty of future job prospects, as well as ChatGPT producing biased results.

Lastly, the three tweets that were identified as neutral were the following:

• “What’s the use case for chatGPT?“

• A professor was caught using ChatGPT to grade final exams. 🤣🤣🤣”

• “For a long time, blue collar jobs were the only one impacted by automation. Now it’s coming for academia.”

While the first two neutral tweets above are fairly neutral, as one is a question and one points out a funny situation, the last “neutral” tweet definitely has more of a negative outlook on ChatGPT. This was clearly not picked up by the sentiment analysis, and this may be because sentiment analysis only scores each individual word, and thus is unable to pick up on the general connotation of a sentence.

The sentiment scores for the tweets are generally accurate, but there are clear errors as the scores for each word do not necessarily take into account the meaning of a sentence when those words are put together.

During such early stages of product-testing ChatGPT and its functionalities, it seems that the public’s impressions are generally positive. However, this doesn’t mean that the advancement of ChatGPT is only doing good; as seen in the negatively categorized tweets and some neutral sentiments, there are clearly concerns around ChatGPT regarding the uncertainty of the future society as such AI technology continues to develop and uses increase, biases and inaccuracy of the outcomes due to imperfect language models, and potential job displacement. In the next section, we further divide the analysis to specific industries to provide more insights on how each specific industry is responding to ChatGPT.

ChatGPT & Technology Industry

In this section, we will explore how the release of ChatGPT was perceived by the technology industry during the first month of its launch, from November 30, 2022 to December 31, 2022. Since ChatGPT is inherently a tech tool, there are many conversations within the tech world surrounding what such new generative artificial intelligence technology means for the future of the tech industry. We performed data analysis on the subset of 12,893 tweets from our original dataset that were related to the tech industry, which was done by pulling tweets from our main dataset that contain any of the keywords “tech,” “technology,” “software,” “developer,” or “developers”.

Word cloud generated from tweets related to ChatGPT and technology

The world cloud above, which was generated by Voyant Tools, gives us a glimpse of what kinds of words people are mentioning in relation to ChatGPT within the tech industry. There are words such as “new” and “future,” which points to the fact that people may be thinking about how ChatGPT is changing the tech industry, as well as concepts such as “artificial intelligence” and “machine learning,” which demonstrate that many people are talking about ChatGPT’s technical background and its affect on these specific fields within computer science. Lastly, we see words such as “use,” and “using,” which shows that users are discussing the ways in which ChatGPT can be utilized. Here are a few tweets relating to the tech industry that contain “use” and “using,” many of which have a positive sentiment:

• “ChatGPT is incredible at teaching technical concepts I’m finding. I would much rather use it than Google when I have a relatively broad technical question.”

• “The impact of ChatGPT on day-to-day software engineering is going to be MASSIVE. Searching for code snippets and information using Google, Stackoverflow, blogs, and documentation will be replaced by chatting with an AI inside your IDE, probably in 2023. And that’s just the start.

• “The potential of AI to revolutionize industries and improve our lives is limitless. \n\nAs we continue to advance in this field, it is crucial to consider the ethical implications of AI and ensure its responsible and transparent development and use.\n\n#softwaredevelopment #ChatGPT”

Within the tech industry, it seems that many people are impressed by ChatGPT’s capability, the ways it can be used, and its potential for the future, as seen by the most used words within tweets relating to the industry. However, many also recognize the ethical concerns that generative AI such as ChatGPT raises and that we must be held accountable for how we allow it to be used.

Through sentiment analysis, we can get a better picture of how people within the tech industry feel about ChatGPT, specifically, what specific sentiments are most common. The above visualization was created using the Syuzhet package in R for sentiment analysis. Looking at the visualization, we see that the sentiment is more polarized, although slightly more positive than negative; the highest sentiment score is positive, followed closely by negative. The next highest sentiment score is ‘trust’, which is followed by ‘fear’. After that, we see that there is ‘anticipation’ for ChatGPT, but this sentiment score is only slightly higher than the score for ‘anger’. Overall, there is generally a positive sentiment towards ChatGPT in the tech field, although the negative sentiment cannot be completely ignored.

Breakdown of Sentiment in the Technology Sector by Compound Score

In order to get a more holistic view of how the sentiment in our technology subset compares, we created a donut chart in R to visually depict how many of the tweets were categorized as positive, how many were negative, and how many were neutral. We found that the proportion of tweets with positive sentiment (59.28%) greatly outweighs the proportion of negative tweets (17.54%) and neutral tweets (23.18%). This chart seems to give a more positive impression of ChatGPT in the tech space as opposed to the impression we get from the sentiment bar graph above, which depicted more polarizing negative sentiments. While this might seem slightly conflicting, the polarizing sentiments are understandable, as with many new technological releases, there are many enthusiasts as well as skeptics. Skeptics may be worried about how ChatGPT may affect employment as well as the ethical implications of generative AI technology as mentioned earlier. To understand this context further, we researched articles from reputable news sources that would give us insight on these concerns.

In the Forbes article “How ChatGPT And Natural Language Technology Might Affect Your Job If You Are A Computer Programmer,” strategic business and technology advisor to governments and companies, Bernard Marr, discusses whether ChatGPT and generative AI technology is a threat to software engineering jobs. He explains that it was a surprise to many that work as software engineers that ChatGPT is capable of writing code. It can create web pages, applications, and even basic games in languages not limited to Python, C, and Javascript. However, it seems that it is not a threat – yet – as it can only create simple programs that lack complexity, and cannot produce something that gives a competitive edge over other software. As a result, although there is potential that generative AI will affect jobs in the tech industry as it develops, as of now, it is not a threat and is instead a beneficial tool (Marr).

To examine the implications of ChatGPT on the rest of the industry, Erin Griffith and Cade Metz, reporters that specialize in startups and emerging technologies, discuss how the release of ChatGPT, or more broadly, generative artificial intelligence, has shaken up the tech industry in The New York Times article, “A New Area of A.I. Booms, Even Amid the Tech Gloom.” In recent years, nothing has been more exciting within the tech industry than generative artificial intelligence, which is technology that when given a prompt from the user, generates text, images, sounds, and other media. Google, Meta, and other large tech companies have been slow to release AI technology similar to ChatGPT because of the public’s concern when it comes to how AI can harm the public through misinformation, hate speech, and biases. However, smaller startups such as OpenAI, the company that released ChatGPT are less concerned with keeping their brand safe and more eager to get their technology out to the public. The release of ChatGPT and the public’s excitement towards it is sure to encourage more tech companies to get into the field of generative AI and push more technology out quickly, as many venture capitalists are also looking to invest in it (Griffith and Metz). Therefore, while generative AI is the future of technology as it only continues to grow and develop, we must be cautious towards it as it can lead to unwanted ethical dilemmas.

ChatGPT & Marketing Industry

The emergence of ChatGPT at the end of 2022 raised the attention of a lot of individuals in the marketing industry. ChatGPT, along with many other tools, has the ability to generate high-quality social media content, provide customer service based on their personal preferences, and suggest relevant recommendations and thereby improving engagement rate and driving sales in various businesses. These strong features, however, have also led some to fear that AI technology could replace marketing work, leading to job losses and displacement. In this section, we conducted sentiment analysis to analyze users’ perspectives on ChatGPT within the marketing industry.

Out of the 219,294 tweets from the original dataset, we extracted a data subset of 1,850 tweets pertaining to marketing by filtering from our main dataset those tweets that contain the keyword “marketing”. Each tweet was given a compound score (positive: >0, neutral: 0, negative: <0) through R, then imported as an Excel file to calculate the percentages. The double-donut chart above is an illustration of the percentage of people who expressed positive, negative, or neutral sentiments toward ChatGPT. Below is a breakdown of the chart:

• Positive – Purple represents the positive sentiments which summed up to about 57% of the entire tweet. we separated tweets that had a score of 0.5 above as ‘Strongly Positive’ (dark pink) and tweets that had a score that was larger than 0 and less than or equal to 0.5 as ‘Fairy Positive’ (light pink). The second layer of the donut chart shows that the percentages of ‘Fairly Positive’ and ‘Strongly Positive’ had a difference of 18%.

• Neutral – Blue represents neutral sentiments, or tweets with a compound score of 0, which summed up to about 31% of the entire tweet.

• Negative – Orange represents negative sentiments which summed up to about 12% of the entire tweets. Compared to the positive and neutral sentiments, there was a relatively small amount of people within the marketing industry that expressed negative sentiments towards ChatGPT. 

Based on the sentiment analysis, we can observe that the general sentiment towards ChatGPT within the marketing industry is leaning positive. However, after analyzing the top most retweeted tweets, we got directed to some new findings regarding a significant social issue – employment. Below is a tweet with a retweet count of 3,158:

• “AI is going to permanently change how marketing works.\n\nIf you’re not paying attention, you’re going to be out of a job pretty soon.\n\nI played around with ChatGPT (a new AI tool) recently.\n\nHere’s how AI will change marketing forever”

The statement above creates a serious sense of emergency and suggests a potential relationship between ChatGPT and the decrease in the employment rate. This finding leads to a socially important question – Will ChatGPT replace human workers in the marketing field?

Bernard Marr is a best-selling author, popular keynote speaker, and consultant who helps organizations improve their business performance and understand new technologies like AI, big data, and blockchain. In “How Will ChatGPT Affect Your Job If You Work In Advertising and Marketing?”, Marr claims that it is important to note that everything ChatGPT generated is from the pre-existed database and something that has been written before. This suggests that at this stage of technical advancement, ChatGPT is not capable of drafting original products in the same way we do as humans. For example, from a marketer’s perspective, building relationships and understanding the target client’s business goal is critical; however, ChatGPT most likely is not fully capable of communicating strategically and effectively just yet. Moreover, Marr proposes that understanding how those changes in the job market will occur and how those changes will affect us is important (Marr 2).

David Boyle, the director of Audience Strategies, has more than 20 years of experience in using advanced technology to develop strategies in creative industries and audience intelligence. Boyle also has a powerful presence on Linkedin with 5000+ followers and aims to help companies better understand their clients and drive growth. In the news source “From Zero To ChatGPT Hero: How To Harness The Power Of AI In Marketing”, Boyle explores the ways that ChatGPT can be beneficial to marketers. The article begins by discussing how ChatGPT isn’t a replacement for human oversight but instead a tool to power human creativity. It’s important to use ChatGPT with an experimental mentality and be willing to reword the prompts to get the most optimal result that suits the marketer’s needs. Boyle suggested that some of the benefits of using ChatGPT include generating ideas and content at scales, streamlining market research and providing audience insights, and condensing and analyzing large volumes of data. In this article, Boyle also explains some of the potential challenges that may arise when implementing AI-powered chatbots in marketing such as the need for iteration and specifying with their prompts to get the best results, the need for human oversight or maintaining human touch and creativity, and the risk of mistakes — acknowledging that the answer or content ChatGPT generated aren’t always correct since the technology is still developing (Boyle 1). 

ChatGPT & Education Industry

While we found that the public generally seems to react in a positive way towards ChatGPT, we wanted to take a closer look at sentiment towards ChatGPT in specific different industries, as the tool’s capabilities would certainly have a different impact in different sectors. Specifically, one of the questions we wanted to research is if there are differences in how ChatGPT is being perceived and embraced by the marketing, technology, and education sectors.

Breakdown of Sentiment in the Education Sector by Compound Score

Out of the 219,294 tweets from the original dataset, we extracted a data subset of 5,868 tweets pertaining to education and schooling by filtering from our main dataset those tweets that contain any of the following keywords “student”, “teacher”, “education”, “edtech”, and “school”. Upon performing a sentiment analysis of these tweets and categorizing their compound scores as positive, negative, or neutral, we found that the proportion of tweets with positive sentiment (52.78%) greatly outweighs the proportion of negative tweets (25.19%) and neutral tweets (22.03%), depicting that ChatGPT is perceived under a more positive light than negative in the education space as well. The average compound score of tweets pertaining to the usage of ChatGPT in education is 0.162, again indicating a slight positive overall sentiment for this sector. 

When comparing the sentiment towards ChatGPT in the education sector with that of the business and technology sector, they are all leaning towards more positive sentiment than negative, as evident from the percentage of positive tweets for each of these sectors (52% positive for education, 57% positive for business, and 59% positive for technology). However, the difference lies in how positive the sentiment is in each of these areas. The sentiment towards ChatGPT is slightly more positive in the business and tech sectors, as opposed to the education sector, confirming our view that ChatGPT is, in fact, perceived slightly differently in different industries. 

However, one issue we found was that the sentiment scores for each tweet may not be completely accurate. From our Tableau bar chart above depicting the sentiment scores for each tweet in the education sector, we noticed that a good amount of the tweets that got fairly high compound scores actually conveyed neutral or even negative sentiment. Similarly, some of the tweets that were scored negatively actually conveyed positive attitudes towards ChatGPT. But even if we account for potential errors in the sentiment compound scores, since business and technology have a more clearly positive sentiment towards ChatGPT, we can still sense that people are generally reacting more positively towards the tool in these two sectors, as opposed to the education sector. Since the results for the education sector are more borderline, when we take into account any potential errors in the sentiment ratings, we can understand that feelings are highly mixed in this space. As the tool continues to evolve and become more widespread in the education sector, it is highly possible that the presence and usage of ChatGPT for educational purposes, whether by students or teachers, will affect educational outcomes, educational policy, and/or how we, as a society, approach education and modify it to best suit future societal and personal needs. 

In the article “ChatGPT in the Classroom: Here’s What Teachers and Students Are Saying”, author Kayla Jimenez, a USA Today reporter of K-12 education issues, sheds light into how ChatGPT is being used in education and what potential concerns exist about the tool. Various novel use cases for ChatGPT are described, such as creating prompts for debate topics, making study guides to prepare for exams, and creating mock business proposals. She cites results from a national survey commissioned by the Walton Family Foundation of more than 1,000 teachers and 1,000 teenage students as evidence of different use cases of ChatGPT in the education system. The survey results show that the majority of students and teachers are now embracing ChatGPT’s role in the new A.I.-powered era, as opposed to previous studies that show a quite large amount of teachers and students worried about the future. 

While in the initial stages, people understandably felt more skeptical about what the future would hold, the current survey shows that more and more people are warming up to the idea that this is just another tool that can help improve current systems if utilized properly. Through such research, we observe that a shift in sentiment did take place over the past few months in the education space, proving that this AI technology is, in fact, influencing how we do things in different segments of society. While concerns exist about potential harms of the tool especially because it’s new and we have not yet fully understood the power it holds, people do continue to discover unique ways to benefit from the tool. In an AI-powered world today, it would be in our best interest to learn to effectively and ethically utilize the tool, while simultaneously gaining awareness about potential harms and working to mitigate them.

ChatGPT & Employment

ChatGPT is seen as an effective tool to assist workers in performing their job duties. However, people are also concerned that the powerful abilities of ChatGPT may threaten workers by replacing them. By using text and sentiment analysis, we investigated whether people view ChatGPT positively or negatively in regard to the topic of employment on Twitter when it first launched. The following analysis was done with a data subset about employment by pulling out tweets that contain the keywords “employment” and “job” from our main dataset.

This is a bar chart created on Tableau visualizing the sentiment of ChatGPT in relation to employment on Twitter in the first month of its launch. It shows the distribution of compound scores, generated in R using Vader, among the 4434 tweets related to employment in our dataset. Red indicates the proportion of negative tweets, gray indicates the proportion of neutral tweets, and blue indicates the proportion of positive tweets. As shown in the visualization, the proportion of tweets with positive sentiment outweighs the proportion of tweets with negative sentiment. There is also a significant number of tweets with neutral sentiments. Computed from our dataset, the average compound score of tweets related to employment is 0.210, which is leaning towards an overall positive sentiment. This informs that people on Twitter generally have a positive sentiment toward ChatGPT in regard to employment.

To support our findings from the compound scores, we have also carried out text analysis by creating a word cloud on Voyant to see what are the most prominent words used in the tweets related to employment during the first month of its launch. The top five words with the highest frequencies in our dataset are “good” (418); “openai” (417); “write” (397); “new” (337); “replace” (327). Majority of the words in the word cloud show a positive sentiment, such as “good,” “great,” “better,” and “new.” There are many action words like “write,” “code,” “make,” “work,” and “help”, which illustrates the ability of ChatGPT and the tasks that people are using ChatGPT for. It also demonstrates the prevalence of using ChatGPT in the workplace.

There are rarely any negative words in the word cloud. However, some neutral terms suggest negative feedback on ChatGPT. For instance, the word “safe” appears in the context of discrediting ChatGPT’s ability in our dataset. Some excerpts of the tweets that contain “safe” are: “Happy to report that chatgpt could not solve my one of my onboarding tasks, so I think my job is safe for now.”; “ChatGPT hasn’t learned how to use Jetpack Compose. Looks like our jobs are safe.”; and “ChatGPT doesn’t know how to use sed properly yet so I think my job is safe for a while.” Some people are questioning the credibility and ability of ChatGPT. They also suggest that people cannot fully rely on ChatGPT for their tasks, and that people should not be too worried about losing their job to ChatGPT.

Overall, this word cloud tells us that people see and use ChatGPT as an useful tool for their jobs. They are using ChatGPT for coding, writing, and creating content. However, there are negative opinions towards ChatGPT that it is not credible and accurate. To summarize, people on Twitter generally see ChatGPT positively in relation to employment. Workers are utilizing ChatGPT for assistance to perform their job duties, including coding and generating content. Employees believe that ChatGPT is a new and effective software that enhances their productivity at work rather than a threat to them because ChatGPT is currently not powerful enough to completely replace labor.

Jaimie Ding from Los Angeles Times further supports our findings about the positive influence of ChatGPT in employment by illustrating in her article that ChatGPT is not only useful for the workforce but also widely utilized by job applicants. In “If you outsource one thing to chatgpt, job-seekers say this should be it”, Ding analyzes how ChatGPT is being used in the process of job application. Ding argues that free public access to ChatGPT has made it faster and more convenient to apply for jobs. She lists that people are now using ChatGPT to write cover letters and review resumes. She supports her argument by interviewing five individuals who successfully landed jobs using ChatGPT. The interviews provide strong evidence that ChatGPT is very powerful and intelligent, as well as how it is now being used pervasively to help job applicants. 

This article offers a positive sentiment towards ChatGPT, which matches the result of our Twitter sentiment analysis toward ChatGPT and employment. It gives us meaningful insights into how ChatGPT is now widely utilized for job applications, as well as the wide-ranging abilities of ChatGPT.

All of these illustrate the importance of ChatGPT in terms of the employment process and work performance. The rise of new AI technologies such as ChatGPT assists people in job applications to better showcase their abilities to employers by polishing their resumes and cover letters, which can ultimately boost employment rates. Moreover, ChatGPT allows workers to perform their tasks more efficiently with its intelligence. The prevalence of using ChatGPT in workplace may also predict future trends of workers and companies implementing chatbots in the workplace for higher efficiency. In conclusion, ChatGPT positively influences the employment process and workers’ productivity, which can be explained by the positive-leaning sentiment toward ChatGPT in relation to employment on Twitter in our findings.