Skip to Main Content

Capital Campus Library

Capital ChatGPT & AI

Limitations

From OpenAI's website, on the blog page titled "ChatGPT: Optimizing Language Models for Dialogue," under Limitations. Emphasis is mine.

  • "ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source of truth; (2) training the model to be more cautious causes it to decline questions that it can answer correctly; and (3) supervised training misleads the model because the ideal answer depends on what the model knows, rather than what the human demonstrator knows.
  • ChatGPT is sensitive to tweaks to the input phrasing or attempting the same prompt multiple times. For example, given one phrasing of a question, the model can claim to not know the answer, but given a slight rephrase, can answer correctly.
  • The model is often excessively verbose and overuses certain phrases, such as restating that it’s a language model trained by OpenAI. These issues arise from biases in the training data (trainers prefer longer answers that look more comprehensive) and well-known over-optimization issues.
  • Ideally, the model would ask clarifying questions when the user provided an ambiguous query. Instead, our current models usually guess what the user intended.
  • While we’ve made efforts to make the model refuse inappropriate requests, it will sometimes respond to harmful instructions or exhibit biased behavior. We’re using the Moderation API to warn or block certain types of unsafe content, but we expect it to have some false negatives and positives for now. We’re eager to collect user feedback to aid our ongoing work to improve this system."

nursing student

Ai-drawn image of female nursing students with warped faces, with a variety of non-white skin colors. They are wearing white uniforms.Craiyon was asked to draw a student in nursing school, a request to which it produced 9 images of white, female nursing students. For the second try, it was asked to draw a minority student in nursing school, after which it produced 9 images of female nursing students with varying shades of skin. This was one of the resulting images. Looking closely, you can see that the faces, including the left eye and teeth of the primary subject, are warped, and her stethoscope is unrealistic.

tortie

AI-drawn tortoiseshell cat, on a green background. The cat has artifacts of two extra eyes above its real ones.Craiyon did better when asked to draw a tortoiseshell cat, as it did not produce a cat wearing a tortoise's shell. However, the creators might want to institute an eye count with the next update.