Python Forum
how to treat two token as one token(lexer)?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
how to treat two token as one token(lexer)?
#1
lets say i have two type of token (the # token and the = token)
when each token is found the position variable will increase by one,but when theres something like joined syntax(not sure what it is called)
example:#=,==,##
the position variable will increase by two ,because it treats=as one token and #as another token
how do i make the the joined syntax increment the position by one instead of two?

i am not on MY computer now ,so i cant show my current python code(the code is in my computer).
so for now,i can only explain.
hope you understand.
sorry for any inconvenient caused.

what i am feeling now: Doh Doh Doh Doh Doh Wall Wall Wall Wall Wall
Reply
#2
This isn't as much a Python issue as a compiler/interpreter issue. You should probably find a forum focused more on that domain than something as generic as the language you're implementing things in.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Unexpected token ';' Azdaghost 3 1,141 Jun-12-2025, 12:37 PM
Last Post: DeaD_EyE
  Get an FFMpeg pass to subprocess.PIPE to treat list as text file? haihal 2 2,310 Nov-21-2024, 11:48 PM
Last Post: haihal
  Refresh token for Wyze SDK duckredbeard 0 2,034 May-16-2022, 04:33 AM
Last Post: duckredbeard
  Need to sign JWT token with JWK key stucoder 1 2,873 Feb-21-2022, 09:04 AM
Last Post: stucoder
  Python requests oauth2 access token herobpv 6 9,146 Sep-27-2021, 06:54 PM
Last Post: herobpv
  unexpected token < in json at position 0 Frodoxzibit 5 4,992 Jul-10-2021, 09:33 AM
Last Post: Larz60+
  Treat error ? JohnnyCoffee 11 7,927 Apr-17-2020, 07:42 AM
Last Post: buran
  / token in function parameterlist tpourjalali 1 2,759 Apr-12-2020, 07:05 PM
Last Post: deanhystad
  [gpxpy] "Error parsing XML: not well-formed (invalid token): line 1, column 1" Winfried 5 9,778 Jan-26-2020, 01:09 AM
Last Post: Winfried
  Getting an "Unexpected Token" Error and don't know why... NotAHackusator 1 2,883 Nov-20-2019, 03:00 PM
Last Post: buran

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020
This forum uses Lukasz Tkacz MyBB addons.
Forum use Krzysztof "Supryk" Supryczynski addons.