PHP token_get_all() Function

The token_get_all() function is an inbuilt function in PHP that is used to tokenize a given PHP source code string into an array of tokens. This function is particularly useful for analyzing, parsing, or manipulating PHP code programmatically.

Syntax:

token_get_all(string $code, int $flags = 0)

Parameters: This function accepts two parameters that are described below.

• $code: This is the string that is going to tokenize.
• $flags: This is an optional parameter that modifies the tokenization behavior of the string.

Return Values: The token_get_all() returns array contains elements where each element represents a token.

Program 1: The following program demonstrates the token_get_all() function.

<?php $code = '<?php echo "Hello, world!"; ?>'; $tokens = token_get_all($code, TOKEN_PARSE); foreach ($tokens as $token) { if (is_array($token)) { echo "Token: " . token_name($token[0]) . " - Content: " . $token[1] . "\n"; } else { echo "Token: " . $token . "\n"; } } ?> 

Token: T_OPEN_TAG - Content: <?php Token: T_ECHO - Content: echo Token: T_WHITESPACE - Content: Token: T_CONSTANT_ENCAPSED_STRING - Content: "Hello, world!" Token: ; Token: T_WHITESPACE - Content: ...

Program 2: The following program demonstrates the token_get_all() function.

<?php $code = '<?php  $message = "Hello, ";  $name = "John";  echo $message . $name; ?>'; $tokens = token_get_all($code); foreach ($tokens as $token) { if (is_array($token)) { list($tokenId, $tokenContent) = $token; echo "Token: " . token_name($tokenId) . " - Content: " . $tokenContent . "\n"; } else { echo "Token: " . $token . "\n"; } } ?> 

Token: T_OPEN_TAG - Content: <?php Token: T_WHITESPACE - Content: Token: T_VARIABLE - Content: $message Token: T_WHITESPACE - Content: Token: = Token: T_WHITESPACE - Content: Token: T_CONSTAN...

Reference: https://www.php.net/manual/en/function.token-get-all.php