Let's build a transpiler! Part 18
This is the eighteenth post in a series of building a transpiler.You can find the previous ones here.
Last time I said we would parse Consts and keep all keywords in a single place.
We'll create a module called Vocabulary and a Property Get for each keyword or contextual keyword we have.
As it is mostly a boring thing to do, I'll just show a snippet instead of the whole module:
Module Vocabulary
Public Property Get vAccess() As String
vAccess = "Access"
End Property
(...)
Public Property Get vXor() As String
vXor = "Xor"
End Property
End Module
Now we will replace every keyword string literal (like "New") for its Vocabulary counterpart (vNew).
You may be wondering why we made this change. Not only we keep track of every keyword or contextual in a single place, but it will allow us to do a crazy thing later.
Also, I could easily be going with a class instead of a module. Maybe I change that later.
Moving on to parsing Consts. Its syntax is [Public | Private] Const name1 [As datatype1] = expression1 [, name2 [As datatype2] = expression2, ...]↩
The problem is we are not ready yet to parse expressions. For now, we will create a function to return whatever the next token is and nothing beyond that.
It is good enough to scan things like "Const PI = 3.14159" but not something like "Const SomeBitPattern = BitMask And Not FirstBitPattern."
Private Function GetExpression() As Token
Set GetExpression = NextToken
End Function
Note that we are not validating what comes from it. We'll change that when we deal properly with expressions.
Next, we will create a class to represent Consts and a place to store them inside Entity:
Class ConstConstruct
Public Access As Accessibility
Public Name As Token
Public DataType As Token
Public Value As Token
End Class
Class Entity
Private Consts_ As Dictionary
Private Sub Class_Initialize()
Set Consts_ = New Dictionary
Consts_.CompareMode = vbTextCompare
End Sub
Public IsClass As Boolean
Public Accessibility As Accessibility
Public Name As Token
Public OptionBase As Integer
Public OptionCompare As VbCompareMethod
Public OptionExplicit As Boolean
Public Property Get Consts() As Dictionary
Set Consts = Consts_
End Property
End Class
Let's adapt Parse to deal with Consts (new code is highlighted:)
Private Function ParseDeclarationArea(ByVal Entity As Entity) As Token
Dim HadOptionBase As Boolean
Dim HadOptionCompare As Boolean
Dim Token As Token
Do
(...)
ElseIf IsKw(Token, vConst) Then
If Access = AccessLocal Then Access = AccessPublic
ParseConsts Access, Entity
Access = AccessLocal
ElseIf IsKw(Token, vEnd) Then
Exit Do
Else
Fail Token, "Expected: Option or Deftype or Public or Private or Const or Enum or Declare or Type"
End If
Loop
Set ParseDeclarationArea = Token
End Function
Let's create some utility function to check if a token is an identifier.
We will also create a function to instantiate a synthetized Token based on the type-declaration character we have in the Sufix property:
Private Function IsId(ByVal Token As Token) As Boolean
IsId = Token.Kind = tkIdentifier Or Token.Kind = tkEscapedIdentifier
End Function
Private Function IsOp(ByVal Token As Token, ByVal Name As String) As Boolean
If Token.Suffix <> vbNullChar Then Exit Function
If Token.Kind <> tkOperator Then Exit Function
IsOp = StrComp(Token.Text, Name, vbTextCompare) = 0
End Function
Private Function FromChar(ByVal TypeDeclarationChar As String) As Token
Dim Token As Token
Set Token = New Token
Token.Kind = tkKeyword
Select Case TypeDeclarationChar
Case "%"
Token.Text = vInteger
Case "&"
Token.Text = vLong
Case "^"
Token.Text = vLongLong
Case "@"
Token.Text = vCurrency
Case "!"
Token.Text = vSingle
Case "#"
Token.Text = vDouble
Case "$"
Token.Text = vString
End Select
Set FromChar = Token
End Function
Finally, let's implement ParseConsts:
Private Sub ParseConsts(ByVal Access As Accessibility, ByVal Entity As Entity)
Const RULE = "[Public | Private] Const identifier [As datatype] = expression (list separator | line break)"
Dim Token As Token
Dim Cnt As ConstConstruct
Do
Rem Get Const's name
Set Token = SkipLineBreaks
If Not IsId(Token) Then Fail Token, RULE, "identifier"
Set Cnt = New ConstConstruct
Cnt.Access = Access
Set Cnt.Name = Token
Set Token = NextToken
Rem Do we have an As clause?
If IsKw(Token, vAs) Then
If Token.Suffix <> vbNullChar Then Fail Token, "Identifier cannot be assigned a new data type"
Rem Get Const's data type name
Set Token = NextToken
If Token.Kind <> tkKeyword Then Fail Token, RULE, "data type"
Select Case Token.Text
Case vBoolean, vByte, vInteger, vLong, vLongLong, vLongPtr, _
vCurrency, vDecimal, vSingle, vDouble, vDate, vString
Rem OK
Case Else
Fail Token, RULE, "data type"
End Select
Set Cnt.DataType = Token
Set Token = NextToken
ElseIf Cnt.Name.Suffix <> vbNullChar Then
Rem Assign DataType property based on type sufix
Set Cnt.DataType = FromChar(Cnt.Name.Suffix)
End If
Rem Discard "="
If Not IsOp(Token, "=") Then Fail Token, RULE, "="
Rem Get Const's value
Set Token = GetExpression
Set Cnt.Value = Token
Rem Ensure it's not a duplicated Const
If Entity.Consts.Exists(Cnt.Name.Text) Then Fail Token, "Ambiguous name detected: " & Cnt.Name.Text
If Cnt.DataType Is Nothing Then
Rem TODO: Infer its data type
End If
Rem Save it
Entity.Consts.Add Cnt.Name.Text, Cnt
Rem Move on
Set Token = NextToken
If IsBreak(Token) Then Exit Do
If Token.Kind <> tkListSeparator Then Fail Token, RULE, "list separator or line break"
Loop
End Sub
For now we can have names that are Tokens. Soon enough we will need to convert them to proper Identifiers.
Also, we left a TODO there in ParseConsts. We are too far away from being able to infer the const's data type yet.
Next week we'll parse Enums.
Andrej Biasic
2020-11-18