Creating a Human-like Chatbot: A Step-by-Step Guide to Training ChatGPT

Paulina Lewandowska

27 Jan 2023
Creating a Human-like Chatbot: A Step-by-Step Guide to Training ChatGPT


It's difficult to create a chatbot that can have appropriate and realistic conversations. The GPT-2 model, which stands for Generative Pre-training Transformer 2, has been refined for conversational tasks after being trained on a vast amount of text data. In this post, we'll go through how to train a ChatGPT (Chat Generative Pre-training Transformer) model so that it may be adjusted to comprehend conversational cues and respond to them in a human-like manner. We'll go into detail about the crucial elements in this approach and how they help to produce a chatbot that can have conversations that flow naturally.

How ChatGPT was made?

ChatGPT is a variant of GPT (Generative Pre-training Transformer), which is a transformer-based language model developed by OpenAI. GPT was trained on a massive dataset of internet text and fine-tuned for specific tasks such as language translation and question answering. GPT-2, an advanced version of GPT, was trained on even more data and has the ability to generate human-like text. ChatGPT is fine-tuned version of GPT-2 to improve its performance in conversational AI tasks.

Training ChatGPT typically involves the following steps:

Collect a large dataset of conversational text, such as transcripts of customer service chats, social media conversations, or other forms of dialog.

What to bear in mind while doing this?

  • The dataset should be large enough to capture a wide variety of conversational styles and topics. The more diverse the data, the better the model will be able to handle different types of input and generate more realistic and appropriate responses.
  • The data should be representative of the types of conversations the model will be used for. For example, if the model will be used in a customer service chatbot, it should be trained on transcripts of customer service chats.
  • If possible, include a variety of different speakers and languages. This will help the model to learn how to generate appropriate responses in different contexts and for different types of users.
  • The data should be diverse in terms of the number of speakers, languages, accents, and cultural background.
  • Label the data with the context of the conversation, such as topic, intent, sentiment, etc.
  • Be sure to filter out any personal information, sensitive data, or any data that could be used to identify a person.

Preprocess the data to clean and format it for training the model. This may include tokenizing the text, removing special characters, and converting the text to lowercase.

A crucial part of training a conversational model like ChatGPT is preprocessing the data. It is beneficial to organize and clean the data so that the model can be trained with ease. Tokenization is the act of dividing the text into smaller parts, like words or phrases, in more detail. This assists in transforming the text into a format that the model can process more quickly. An application like NLTK or SpaCy can be used to perform the tokenization procedure.

Eliminating special characters and changing the text's case are further crucial steps. Converting the text to lowercase helps to standardize the data and lowers the amount of unique words the model needs to learn. Special characters can cause problems while training the model. In data preparation, it's also a good idea to eliminate stop words, which are frequent words like "a," "an," "the," etc. that don't have any significant meaning. It's also a good idea to replace dates or numbers with a specific token like "NUM" or "DATE" when preparing data. In data preparation, it's also a good idea to replace terms that are unknown or not in the model's lexicon with a unique token, such as "UNK." 

It is crucial to note that preparing the data can take time, but it is necessary to make sure the model can benefit from the data. Preprocessing the data makes it easier for the model to interpret and learn from it. It also makes the data more consistent.

Fine-tune a pre-trained GPT-2 model on the conversational dataset using a framework such as Hugging Face's Transformers library.

The procedure entails tweaking the model's hyperparameters and running several epochs of training on the conversational dataset. This can be accomplished by utilizing a framework like Hugging Face's Transformers library, an open-source natural language processing toolkit that offers pre-trained models and user-friendly interfaces for optimizing them.

The rationale behind fine-tuning a pre-trained model is that it has previously been trained on a sizable dataset and has a solid grasp of the language's overall structure. The model can be refined on a conversational dataset so that it can learn to produce responses that are more tailored to the conversation's topic. The refined model will perform better at producing responses that are appropriate for customer service interactions, for instance, if the conversational dataset consists of transcripts of discussions with customer service representatives.

It is important to note that the model's hyperparameters, such as the learning rate, batch size, and number of layers, are frequently adjusted throughout the fine-tuning phase. The performance of the model can be significantly impacted by these hyperparameters, thus it's necessary to experiment with different settings to discover the ideal one. Additionally, depending on the size of the conversational dataset and the complexity of the model, the fine-tuning procedure can need a significant amount of time and processing resources. But in order for the model to understand the precise nuances and patterns of the dialogue and become more applicable to the task, this stage is essential.

Evaluate the model's performance on a held-out test set to ensure it generates realistic and appropriate responses.

A held-out test set, which is a dataset distinct from the data used to train and fine-tune the model, is one popular strategy. The model's capacity to produce realistic and pertinent responses is evaluated using the held-out test set. 

Measuring a conversational model's capacity to provide suitable and realistic responses is a typical technique to assess its performance. This can be achieved by assessing the similarity between the model-generated and human-written responses. Utilizing metrics like BLEU, METEOR, ROUGE, and others is one approach to do this. These metrics assess how comparable the automatically generated and manually written responses are to one another.

Measuring a conversational model's capacity to comprehend and respond to various inputs is another technique to assess its performance. This can be accomplished by putting the model to the test with various inputs and evaluating how well it responds to them. You might test the model using inputs with various intents, subjects, or feelings and assess how effectively it can react.

Use the trained model to generate responses to new input.

Once trained and improved, the model can be utilized to produce answers to fresh input. The last stage in creating a chatbot is testing the model to make sure it can respond realistically and appropriately to new input. The trained model processes the input before producing a response. It's crucial to remember that the caliber of the reaction will depend on the caliber of the training data and the procedure of fine-tuning.

Context is crucial when using a trained model to generate responses in a conversation. To produce responses that are relevant and appropriate to the current conversation, it's important to keep track of the conversation history. A dialogue manager, which manages the conversation history and creates suitable inputs for the model, can be used to accomplish this.

Especially when employing a trained model to generate responses, it's critical to ensure the quality of the responses the model generates. As the model might not always create suitable or realistic responses, a technique for weeding out improper responses should be in place. Using a post-processing phase that would filter out inappropriate responses and choose the best one is one way to accomplish this.


Training a ChatGPT model is a multi-step process that requires a large amount of data. The GPT-2 model with its ability to generate human-like text and fine-tuning it with conversational dataset can lead to very powerful results which might be extremely helpful in everyday life. The process of training is essential in creating a chatbot that can understand and respond to conversational prompts in a natural and seamless manner. As the field of AI continues to evolve, the development of sophisticated chatbots will play an increasingly important role in enhancing the way we interact with technology. Interested? Check out our other articles related to AI!


Most viewed

Never miss a story

Stay updated about Nextrope news as it happens.

You are subscribed

Token Engineering Process

Kajetan Olas

13 Apr 2024
Token Engineering Process

Token Engineering is an emerging field that addresses the systematic design and engineering of blockchain-based tokens. It applies rigorous mathematical methods from the Complex Systems Engineering discipline to tokenomics design.

In this article, we will walk through the Token Engineering Process and break it down into three key stages. Discovery Phase, Design Phase, and Deployment Phase.

Discovery Phase of Token Engineering Process

The first stage of the token engineering process is the Discovery Phase. It focuses on constructing high-level business plans, defining objectives, and identifying problems to be solved. That phase is also the time when token engineers first define key stakeholders in the project.

Defining the Problem

This may seem counterintuitive. Why would we start with the problem when designing tokenomics? Shouldn’t we start with more down-to-earth matters like token supply? The answer is No. Tokens are a medium for creating and exchanging value within a project’s ecosystem. Since crypto projects draw their value from solving problems that can’t be solved through TradFi mechanisms, their tokenomics should reflect that. 

The industry standard, developed by McKinsey & Co. and adapted to token engineering purposes by Outlier Ventures, is structuring the problem through a logic tree, following MECE.
MECE stands for Mutually Exclusive, Collectively Exhaustive. Mutually Exclusive means that problems in the tree should not overlap. Collectively Exhaustive means that the tree should cover all issues.

In practice, the “Problem” should be replaced by a whole problem statement worksheet. The same will hold for some of the boxes.
A commonly used tool for designing these kinds of diagrams is the Miro whiteboard.

Identifying Stakeholders and Value Flows in Token Engineering

This part is about identifying all relevant actors in the ecosystem and how value flows between them. To illustrate what we mean let’s consider an example of NFT marketplace. In its case, relevant actors might be sellers, buyers, NFT creators, and a marketplace owner. Possible value flow when conducting a transaction might be: buyer gets rid of his tokens, seller gets some of them, marketplace owner gets some of them as fees, and NFT creators get some of them as royalties.

Incentive Mechanisms Canvas

The last part of what we consider to be in the Discovery Phase is filling the Incentive Mechanisms Canvas. After successfully identifying value flows in the previous stage, token engineers search for frictions to desired behaviors and point out the undesired behaviors. For example, friction to activity on an NFT marketplace might be respecting royalty fees by marketplace owners since it reduces value flowing to the seller.


Design Phase of Token Engineering Process

The second stage of the Token Engineering Process is the Design Phase in which you make use of high-level descriptions from the previous step to come up with a specific design of the project. This will include everything that can be usually found in crypto whitepapers (e.g. governance mechanisms, incentive mechanisms, token supply, etc). After finishing the design, token engineers should represent the whole value flow and transactional logic on detailed visual diagrams. These diagrams will be a basis for creating mathematical models in the Deployment Phase. 

Token Engineering Artonomous Design Diagram
Artonomous design diagram, source: Artonomous GitHub

Objective Function

Every crypto project has some objective. The objective can consist of many goals, such as decentralization or token price. The objective function is a mathematical function assigning weights to different factors that influence the main objective in the order of their importance. This function will be a reference for machine learning algorithms in the next steps. They will try to find quantitative parameters (e.g. network fees) that maximize the output of this function.
Modified Metcalfe’s Law can serve as an inspiration during that step. It’s a framework for valuing crypto projects, but we believe that after adjustments it can also be used in this context.

Deployment Phase of Token Engineering Process

The Deployment Phase is final, but also the most demanding step in the process. It involves the implementation of machine learning algorithms that test our assumptions and optimize quantitative parameters. Token Engineering draws from Nassim Taleb’s concept of Antifragility and extensively uses feedback loops to make a system that gains from arising shocks.

Agent-based Modelling 

In agent-based modeling, we describe a set of behaviors and goals displayed by each agent participating in the system (this is why previous steps focused so much on describing stakeholders). Each agent is controlled by an autonomous AI and continuously optimizes his strategy. He learns from his experience and can mimic the behavior of other agents if he finds it effective (Reinforced Learning). This approach allows for mimicking real users, who adapt their strategies with time. An example adaptive agent would be a cryptocurrency trader, who changes his trading strategy in response to experiencing a loss of money.

Monte Carlo Simulations

Token Engineers use the Monte Carlo method to simulate the consequences of various possible interactions while taking into account the probability of their occurrence. By running a large number of simulations it’s possible to stress-test the project in multiple scenarios and identify emergent risks.

Testnet Deployment

If possible, it's highly beneficial for projects to extend the testing phase even further by letting real users use the network. Idea is the same as in agent-based testing - continuous optimization based on provided metrics. Furthermore, in case the project considers airdropping its tokens, giving them to early users is a great strategy. Even though part of the activity will be disingenuine and airdrop-oriented, such strategy still works better than most.

Time Duration

Token engineering process may take from as little as 2 weeks to as much as 5 months. It depends on the project category (Layer 1 protocol will require more time, than a simple DApp), and security requirements. For example, a bank issuing its digital token will have a very low risk tolerance.

Required Skills for Token Engineering

Token engineering is a multidisciplinary field and requires a great amount of specialized knowledge. Key knowledge areas are:

  • Systems Engineering
  • Machine Learning
  • Market Research
  • Capital Markets
  • Current trends in Web3
  • Blockchain Engineering
  • Statistics


The token engineering process consists of 3 steps: Discovery Phase, Design Phase, and Deployment Phase. It’s utilized mostly by established blockchain projects, and financial institutions like the International Monetary Fund. Even though it’s a very resource-consuming process, we believe it’s worth it. Projects that went through scrupulous design and testing before launch are much more likely to receive VC funding and be in the 10% of crypto projects that survive the bear market. Going through that process also has a symbolic meaning - it shows that the project is long-term oriented.

If you're looking to create a robust tokenomics model and go through institutional-grade testing please reach out to Our team is ready to help you with the token engineering process and ensure your project’s resilience in the long term.


What does token engineering process look like?

  • Token engineering process is conducted in a 3-step methodical fashion. This includes Discovery Phase, Design Phase, and Deployment Phase. Each of these stages should be tailored to the specific needs of a project.

Is token engineering meant only for big projects?

  • We recommend that even small projects go through a simplified design and optimization process. This increases community's trust and makes sure that the tokenomics doesn't have any obvious flaws.

How long does the token engineering process take?

  • It depends on the project and may range from 2 weeks to 5 months.

What is Berachain? 🐻 ⛓️ + Proof-of-Liquidity Explained


18 Mar 2024
What is Berachain? 🐻 ⛓️ + Proof-of-Liquidity Explained

Enter Berachain: a high-performance, EVM-compatible blockchain that is set to redefine the landscape of decentralized applications (dApps) and blockchain services. Built on the innovative Proof-of-Liquidity consensus and leveraging the robust Polaris framework alongside the CometBFT consensus engine, Berachain is poised to offer an unprecedented blend of efficiency, security, and user-centric benefits. Let's dive into what makes it a groundbreaking development in the blockchain ecosystem.

What is Berachain?


Berachain is an EVM-compatible Layer 1 (L1) blockchain that stands out through its adoption of the Proof-of-Liquidity (PoL) consensus mechanism. Designed to address the critical challenges faced by decentralized networks. It introduces a cutting-edge approach to blockchain governance and operations.

Key Features

  • High-performance Capabilities. Berachain is engineered for speed and scalability, catering to the growing demand for efficient blockchain solutions.
  • EVM Compatibility. It supports all Ethereum tooling, operations, and smart contract languages, making it a seamless transition for developers and projects from the Ethereum ecosystem.
  • Proof-of-Liquidity.This novel consensus mechanism focuses on building liquidity, decentralizing stake, and aligning the interests of validators and protocol developers.


EVM-Compatible vs EVM-Equivalent


EVM compatibility means a blockchain can interact with Ethereum's ecosystem to some extent. It can interact supporting its smart contracts and tools but not replicating the entire EVM environment.


An EVM-equivalent blockchain, on the other hand, aims to fully replicate Ethereum's environment. It ensures complete compatibility and a smooth transition for developers and users alike.

Berachain's Position

Berachain can be considered an "EVM-equivalent-plus" blockchain. It supports all Ethereum operations, tooling, and additional functionalities that optimize for its unique Proof-of-Liquidity and abstracted use cases.

Berachain Modular First Approach

At the heart of Berachain's development philosophy is the Polaris EVM framework. It's a testament to the blockchain's commitment to modularity and flexibility. This approach allows for the easy separation of the EVM runtime layer, ensuring that Berachain can adapt and evolve without compromising on performance or security.

Proof Of Liquidity Overview

High-Level Model Objectives

  • Systemically Build Liquidity. By enhancing trading efficiency, price stability, and network growth, Berachain aims to foster a thriving ecosystem of decentralized applications.
  • Solve Stake Centralization. The PoL consensus works to distribute stake more evenly across the network, preventing monopolization and ensuring a decentralized, secure blockchain.
  • Align Protocols and Validators. Berachain encourages a symbiotic relationship between validators and the broader protocol ecosystem.

Proof-of-Liquidity vs Proof-of-Stake

Unlike traditional Proof of Stake (PoS), which often leads to stake centralization and reduced liquidity, Proof of Liquidity (PoL) introduces mechanisms to incentivize liquidity provision and ensure a fairer, more decentralized network. Berachain separates the governance token (BGT) from the chain's gas token (BERA) and incentives liquidity through BEX pools. Berachain's PoL aims to overcome the limitations of PoS, fostering a more secure and user-centric blockchain.

Berachain EVM and Modular Approach

Polaris EVM

Polaris EVM is the cornerstone of Berachain's EVM compatibility, offering developers an enhanced environment for smart contract execution that includes stateful precompiles and custom modules. This framework ensures that Berachain not only meets but exceeds the capabilities of the traditional Ethereum Virtual Machine.


The CometBFT consensus engine underpins Berachain's network, providing a secure and efficient mechanism for transaction verification and block production. By leveraging the principles of Byzantine fault tolerance (BFT), CometBFT ensures the integrity and resilience of the Berachain blockchain.


Berachain represents a significant leap forward in blockchain technology, combining the best of Ethereum's ecosystem with innovative consensus mechanisms and a modular development approach. As the blockchain landscape continues to evolve, Berachain stands out as a promising platform for developers, users, and validators alike, offering a scalable, efficient, and inclusive environment for decentralized applications and services.


For those interested in exploring further, a wealth of resources is available, including the Berachain documentation, GitHub repository, and community forums. It offers a compelling vision for the future of blockchain technology, marked by efficiency, security, and community-driven innovation.


How is Berachain different?

  • It integrates Proof-of-Liquidity to address stake centralization and enhance liquidity, setting it apart from other blockchains.

Is Berachain EVM-compatible?

  • Yes, it supports Ethereum's tooling and smart contract languages, facilitating easy migration of dApps.

Can it handle high transaction volumes?

  • Yes, thanks to the Polaris framework and CometBFT consensus engine, it's built for scalability and high throughput.