Implement strtok() function in C++

The strtok() function is one of the most commonly used functions in C. Using delimiters as a guide, this function can split text into smaller chunks or tokens. Working with strings in C is easy thanks to the strtok() function. This article will explain the strtok() function in detail, including its definition, syntax, algorithm and various implementation strategies. Something to keep in mind is that the strtok function has some limitations and potential drawbacks. For example, it cannot be used with const or read-only strings because it directly changes the original string. Edge cases and unexpected input, such as empty strings or delimiters that recur in a sequence, can also be difficult to handle.

Despite these shortcomings, this function remains a useful tool for many programmers and is often used in a wide variety of applications, including text processing, data parsing, and network protocols. It is a flexible and powerful function that can significantly simplify many common programming tasks.

Characteristics of strtok() function

To split a string into smaller chunks or tokens based on a delimiter, use the strtok() function in C . A pointer to the string that has to be tokenized and a string with the delimiter characters are the two inputs for the function . A pointer to the string's very first token is returned by the function. As long as there are tokens left in the string, successive calls to the function with the same string parameter will return additional tokens and this happens with the help of NULL pointer in while loop. When calling the'strtok()' function again, supplying NULL as the first argument instructs the function to pick up where it left off with the previous tokenization of the same string.

When the function'strtok()' is first called with the input string as the first argument, it starts looking for the first token in the string at the beginning. When it locates the initial token, it terminates it by replacing the delimiter that comes after it with the null character (''0''). When'strtok()' is used again with a NULL pointer as the first argument, the function picks up where it left off with the previous tokenization of the string , that is, directly after the previous token. It keeps looking for the delimiter's next appearance.

grammar

Char* strtok(char* str, const char* delimiters);  

Where

- `str` is a pointer to the string to be tokenized.

- `delimiters` is a string containing delimiter characters.

algorithm

• Step 1 − The `strtok()` function takes two arguments - the first argument is the input string to be tokenized and the second argument is a string containing all the delimiters that should be used to split the input string into tokens.

• Step 2 − When the function is called for the first time, the input string is passed as the first argument, and the delimiter string is passed as the second argument.

• Step 3 − This function searches the input string for the first occurrence of any delimiter character

• Step 4 − When it finds a delimiter character, it replaces it with a null terminator (`'\0''`) and returns a pointer to the beginning of the first token.

• Step 5 - The next time the function is called, the first parameter should be set to `NULL` instead of the original input string. This tells the function to continue where it left off and search for the next token in the string.

• Step 6 − The function continues to search for delimiter characters and returns pointers to the beginning of subsequent tokens until it reaches the end of the string, at which point it returns `NULL`.

Approaches to follow

Method 1 − A program that uses loops and strtok() function and a single delimiter to display the code.

Method 2 - Program to display code using loops and strtok() function and multiple delimiters

method one

Below is the program to show code using a loop with strtok() function and single delimeter

This example demonstrates how to use strtok to segment a string by spaces. The input string str is first passed to strtok along with the delimiter " ". The first call to strtok returns a pointer to the first token ("Hi,"), which is printed to the console. strtok continues tokenizing the string until there are no more tokens and each token is printed to the console. Please note that for subsequent calls to strtok, we pass a null pointer as the first parameter, indicating that we want to continue tokenizing the original string

Example 1

#include <iostream>
#include <cstring> // Include the header for strtok()
using namespace std;

int main() {
   char str[] = "Hi, this topic tells about strtok() function";
   char* ptr;
   ptr = strtok(str, " ");
   while (ptr != NULL) {
      cout << ptr << endl;
      ptr = strtok(NULL, " ");
   }
   return 0;
}

Output

Hi, 
this  
topic 
tells  
about 
strtok()  
function 

Method Two

Below is a program that uses loops and strtok() function to display code with multiple delimiters. Below is the program code for the same.

The '|' and'' (space) delimiters are used to tokenize the string "The boy|is|standing|with his pet|lazy dog." in this example. Once with the string as the first argument and the characters '|' and'' as the second argument, the strtok() method is called twice. Following calls return the remaining tokens, the initial call returns the first token, "The." Until all tokens have been extracted, the loop keeps running.

Example-2

#include <iostream> 
#include <cstring> 
 
int main() { 
   char str[] = "The boy|is|standing|with his pet|lazy dog."; 
   char* token = strtok(str, "| "); 
   while (token != NULL) { 
      std::cout << token << std::endl; 
      token = strtok(NULL, "| "); 
   } 
   return 0; 
} 

Output

The 
boy 
is 
standing 
with 
his 
pet 
lazy 
dog.

结论

总之，C++的strtok函数是一个有用且有效的用于操作字符串的工具。尽管它有一些重要的限制和潜在的缺点，但它仍然是解析和处理文本数据的常见选择，并且对于任何编码人员来说都是一个有用的工具。

The above is the detailed content of Implement strtok() function in C++. For more information, please follow other related articles on the PHP Chinese website!