Question

In: Computer Science

C++ Question The first phase of compilation is called scanning or lexical analysis. This phase interprets...

C++ Question

The first phase of compilation is called scanning or lexical analysis. This phase interprets the input program as a sequence of characters and produces a sequence of tokens, which will be used by the parser.

Write a C++ program that implements a simple scanner for a source file given as a command-line argument. The format of the tokens is described below. You may assume that the input is syntactically correct. Optionally, your program can build a symbol table (a hash table is a good choice), which contains an entry for each token that was found in the input. When all the input has been read, your program should produce a summary report that includes a list of all the tokens that appeared in the input, the number of times each token appears in the input and the class of each token. Your program should also list how many times tokens of each category appeared in the input.

Sample token format:

        keyword -> if | then | else | begin | end
        identifier -> character | character identifier
        integer -> digit | digit integer
        real -> integer.integer
        special -> ( | ) | [ | ] | + | - | = | , | ;
        digit -> 0|1|2|3|4|5|6|7|8|9
        character -> a|b|c ... |z|A|B|C ... |Z

More details:

  • Case is not used to distinguish tokens.
  • The delimiters can be space, tab, newline, and special characters. You only need to use one delimiter in your implementation.
  • Choice at least 5 token classes or use the sample ones above.
  • Your program output can be ordered in any way you want.

Solutions

Expert Solution

The following program tokenizes a given source file and classifies the them into five different classes namely Keywords, Real, Special, Character and Digit. To add more items to a class, just modify and add the items to the unordered_set variables defined at the top of the file. Compile with the -std flga set to c++11

#include <iostream>
#include <fstream>
#include <cstdlib>
#include <unordered_map>
#include <unordered_set>
#include <vector>
#include <string>
#include <locale>

using namespace std;

unordered_set<string> keywords = {"if","then","else","begin","end"};
unordered_set<string> special = {"(",")","[","]","+","-","=",",",";"};
unordered_set<string> digit_set = {"0","1","2","3","4","5","6","7","8","9"};

bool isKeyword(string key) {
return keywords.find(key) != keywords.end();
}

bool isSpecial(string key) {
return special.find(key) != special.end();
}

bool isDigit(string key) {
return digit_set.find(key) != digit_set.end();
}

bool isChar(string key) {
return (key.length() == 1 && (key.at(0) >= 'A' && key.at(0) <= 'Z' && key.at(0) >= 'a' && key.at(0) <= 'z'));
}

bool isReal(string key) {
int found = key.find(".");
if(found != string::npos) {
string part1 = key.substr(0,found-1);
string part2 = key.substr(found+1,key.length());
if(isDigit(part1) && isDigit(part2))
return true;
}
return false;
}

int main(int argc, char *argv[]) {
if(argc < 2) {
cout << "Incorrect usage! use: ./tokenize filename.ext";
exit(-1);
}

ifstream fin;
fin.open(argv[1]);
unordered_map<string,vector<string>> symbol_table;
vector<string> keyword_vec, real_vec, special_vec, character_vec, digit_vec;
symbol_table["keyword"] = keyword_vec;
symbol_table["real"] = real_vec;
symbol_table["special"] = special_vec;
symbol_table["character"] = character_vec;
symbol_table["digit"] = digit_vec;

string temp;
while(fin) {
getline(fin,temp,' ');
if(isKeyword(temp))
symbol_table["keyword"].push_back(temp);
else if(isReal(temp))
symbol_table["real"].push_back(temp);
else if(isSpecial(temp))
symbol_table["special"].push_back(temp);
else if(isChar(temp))
symbol_table["character"].push_back(temp);
else if(isDigit(temp))
symbol_table["digit"].push_back(temp);
}
fin.close();
cout << "\n-----File Summary------";
cout << "\n-----------------------";
cout << "\nKeywords : \t" << symbol_table["keyword"].size();
cout << "\nReal Values : \t" << symbol_table["real"].size();
cout << "\nSpecials : \t" << symbol_table["special"].size();
cout << "\nCharacters : " << symbol_table["character"].size();
cout << "\nDigits : " << symbol_table["digit"].size();
return 0;
}


Related Solutions

C++ Question The first phase of compilation is called scanning or lexical analysis. This phase interprets...
C++ Question The first phase of compilation is called scanning or lexical analysis. This phase interprets the input program as a sequence of characters and produces a sequence of tokens, which will be used by the parser. Write a C++ program that implements a simple scanner for a source file given as a command-line argument. The format of the tokens is described below. You may assume that the input is syntactically correct. Optionally, your program can build a symbol table...
C++ Question The first phase of compilation is called scanning or lexical analysis. This phase interprets...
C++ Question The first phase of compilation is called scanning or lexical analysis. This phase interprets the input program as a sequence of characters and produces a sequence of tokens, which will be used by the parser. Write a C++ program that implements a simple scanner for a source file given as a command-line argument. The format of the tokens is described below. You may assume that the input is syntactically correct. Optionally, your program can build a symbol table...
The objective of this assignment is to gain an understanding of the lexical analysis phase of...
The objective of this assignment is to gain an understanding of the lexical analysis phase of a compiler and the process of constructing a symbol table. Problem: The first phase of compilation is called scanning or lexical analysis. This phase interprets the input program as a sequence of characters and produces a sequence of tokens, which will be used by the parser. Write a program (in C, C++, C#, Java, or Python) that implements a simple scanner for a source...
Question: Analysis and Design Models: What's the Difference? With the analysis phase behind us, reflect on...
Question: Analysis and Design Models: What's the Difference? With the analysis phase behind us, reflect on the models built for the Theater project. How are they different from the models you plan on using during the design phase. Explain the differences in your own words. Next, find an article and/or video to share with the class on the subject.  
QUESTION 2 You are a Chief Technical Officer (CTO) who is now leading the first phase...
QUESTION 2 You are a Chief Technical Officer (CTO) who is now leading the first phase of implementation of ERP SAP system in your company. The ERP SAP system will bring significant changes to the business processes. Eighty percent of the employees of your company are aged between 40 to 60 years old, with an average of ten years working experiences with the company. a) Based on your knowledge gained in this ERP course, how would the employees react to...
Question 63.33 pts What is the first block in a blockchain called? Group of answer choices...
Question 63.33 pts What is the first block in a blockchain called? Group of answer choices Genesis block First block Alpha block Initial block Question 53.33 pts Data is permanently recorded on the Bitcoin network through files called Group of answer choices Blocks Nounce Node Safety file
C++ program homework question 1 1. Create and implement a class called clockType with the following...
C++ program homework question 1 1. Create and implement a class called clockType with the following data and methods (60 Points.): Data: Hours, minutes, seconds Methods: Set and get hours Set and get minutes Set and get seconds printTime(…) to display time in the form of hh:mm:ss default and overloading constructor Overloading Operators: << (extraction) operator to display time in the form of hh:mm:ss >> (insertion) operator to get input for hours, minutes, and seconds operator+=(int x) (increment operator) to...
Hello, this question relates to a class I am taking called introduction to C++. I have...
Hello, this question relates to a class I am taking called introduction to C++. I have no experience writing programs and outside of learning out of a textbook, and studying on my own, have little understanding of logic. I have been assigned a problem that requires me to write a program for a Grocery Bill, where the consumer inputs the price for 5 items, that the program calculates the total with a 6% sales tax. I really am not sure...
hello i have question in c++ language Q1: create a class called RightTriangleShape, and it has...
hello i have question in c++ language Q1: create a class called RightTriangleShape, and it has data member called height which initialized to 3 by the constructor of the class. It has also the following function members: Void setHeight() to read, and set the height. Void getHeight() to print the value of height. void leftBottom_RTraingle(), prints shape a void leftTop_RTraingle(), prints shape b. void RightTop_RTraingle(), prints shape c. void RigtBottom_RTraingle(), prints shape d. The functions from 3 to 6 have...
AM REALLY IN NEED OF THE NARRATIVE ANALYSIS PART Question 1: Your first task is to...
AM REALLY IN NEED OF THE NARRATIVE ANALYSIS PART Question 1: Your first task is to determine whether your firm is in a competitive industry. Based on the following demand function for the firm's product, what would you answer? Q = 50,000 – 25*P Q is the amount produced and P is the price. . Submit your Competitive Industry Report and Calculations to the dropbox below. Be sure to show your calculations in Excel and provide a narrative analysis in...
ADVERTISEMENT
ADVERTISEMENT
ADVERTISEMENT