orion

Introduction#

REPL (Read-Eval-Print Loop) is an interactive programming environment that allows users to input code and immediately see the results. In this article, we will introduce how to write a simple REPL in Rust that can read user-inputted code and print the results in the console.

Implementation Steps#

This REPL is mainly divided into three steps: reading user input, parsing user input, and executing user input.

1. Reading User Input

   First, we need to loop to read user input. When the user presses the Enter key, we can obtain the content the user has entered in the console. If the user has not entered anything, we can continue to wait for user input.

2. Parsing User Input

   Once we have obtained the user input, we need to parse it into a format that the program can understand. We first check whether the user input is a meta-command (starting with "."), and if so, we execute the corresponding meta-command. If it is not a meta-command, we need to parse the SQL statement entered by the user.

3. Executing User Input

   After parsing the SQL statement entered by the user, we can execute this statement. We first need to preprocess the input SQL statement and then execute it. If the execution is successful, we can print the corresponding results in the console. If the execution fails, we need to print the error message in the console.

The above is the basic flow of this REPL. If you want to know more details, please check the code.

Code#

main.rs#

mod executor;
mod parser;
use executor::{execute_statement, PrepareError, Statement};
use parser::{do_meta_command, prepare_statement, MetaCommandResult};
use std::io::{self, Write};

fn main() -> io::Result<()> {
    loop {
        print_prompt();
        io::stdout().flush()?;
        let mut cmd = String::new();
        io::stdin().read_line(&mut cmd)?;
        if cmd.trim().is_empty() {
            continue;
        }

        if cmd.starts_with('.') {
            match do_meta_command(cmd.trim()) {
                MetaCommandResult::Exit => {
                    println!("exit, bye");
                    break;
                }
                MetaCommandResult::UnrecognizedCommand => {
                    println!("Unrecognized command '{}'", cmd.trim());
                    continue;
                }
            };
        }

        let mut stmt: Statement = Statement::default();

        match prepare_statement(cmd.trim(), &mut stmt) {
            Ok(_) => {}

            Err(PrepareError::UnrecognizedStatement) => {
                println!("Unrecognized keyword at start of '{}'", cmd.trim());
                continue;
            }
        };

        match execute_statement(&stmt) {
            Ok(_) => {
                println!("Executed.");
            }
            Err(_) => {
                println!("Error executing statement");
                continue;
            }
        }
    }

    Ok(())
}

fn print_prompt() {
    print!("db > ")
}

The prepare_statement function is encapsulated in parser.rs because its role is to parse the SQL statement entered by the user into a format that the program can understand, which falls under the category of syntax analysis. Therefore, it is more appropriate to place it in the parser.rs file; while the execute_statement function is responsible for executing the parsed statement, which needs to be completed by the executor, so it is encapsulated in the executor.rs file.

parser.rs#

use crate::executor::{PrepareError, Statement, StatementType};
pub enum MetaCommandResult {
    Exit,
    UnrecognizedCommand,
}

pub fn do_meta_command(cmd: &str) -> MetaCommandResult {
    if cmd == ".exit" {
        MetaCommandResult::Exit
    } else {
        MetaCommandResult::UnrecognizedCommand
    }
}

pub fn prepare_statement(cmd: &str, stmt: &mut Statement) -> Result<(), PrepareError> {
    if cmd.starts_with("insert") {
        stmt.statement_type = StatementType::Insert;
        // Parse cmd into row
        let mut iter = cmd.split_whitespace();
        // Check if there are three parameters
        if iter.clone().count() != 4 {
            return Err(PrepareError::UnrecognizedStatement);
        }
        iter.next();

        stmt.row_to_insert.id = iter.next().unwrap().parse::<u32>().unwrap();
        stmt.row_to_insert.username = iter.next().unwrap().to_string();
        stmt.row_to_insert.email = iter.next().unwrap().to_string();
        println!("stmt.row_to_insert: {:?}", stmt.row_to_insert);

        return Ok(());
    }
    if cmd.starts_with("select") {
        stmt.statement_type = StatementType::Select;
        return Ok(());
    }
    Err(PrepareError::UnrecognizedStatement)
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_prepare_insert() {
        let mut stmt = Statement::default();
        let cmd = "insert 1 user1 [email protected]";
        let _result = prepare_statement(cmd, &mut stmt);
        if stmt.statement_type != StatementType::Insert {
            panic!("must be insert statement");
        }

        // Test if each field is correct
        assert_eq!(stmt.row_to_insert.id, 1);
        assert_eq!(stmt.row_to_insert.username, "user1");
        assert_eq!(stmt.row_to_insert.email, "[email protected]");
    }
}

This is the parser.rs file in the code, which contains functions for parsing user input and meta-commands. Meta-commands are commands that start with ".", used to control the behavior of the REPL, such as exiting the REPL.

The do_meta_command function is used to parse meta-commands. If the user inputs ".exit", the function returns MetaCommandResult::Exit, indicating that the REPL should exit; if the user inputs other meta-commands, the function returns MetaCommandResult::UnrecognizedCommand, indicating that the meta-command is unrecognized.

The prepare_statement function is used to parse the SQL statement entered by the user into a format that the program can understand. If the user inputs "insert", the function parses the SQL statement into a Statement struct that contains the fields of the row to be inserted. If the user inputs "select", the function parses the SQL statement into a Statement struct without any query conditions. If the user inputs other statements, the function returns PrepareError::UnrecognizedStatement, indicating that the statement is unrecognized.

The code in the prepare_statement function implements the logic for parsing "insert" statements. First, it sets the statement_type field of the Statement struct to StatementType::Insert, indicating that a row of data is to be inserted. Then, it parses the SQL statement into an iterator, using the split_whitespace function to split it into words. If the number of words is not 4, the function returns PrepareError::UnrecognizedStatement, indicating that the statement is unrecognized. Otherwise, the function assigns the parsed values to the row_to_insert field of the stmt struct, including the id, username, and email fields.

In the test, we used test_prepare_insert to test whether the prepare_statement function can correctly parse "insert" statements and assign the parsed values to the row_to_insert field of the stmt struct.

executor.rs#

#![allow(dead_code)]

use serde::{Deserialize, Serialize};
use std::fmt::Display;

#[derive(Debug, Eq, PartialEq)]
pub enum StatementType {
    Insert,
    Select,
    Unrecognized,
}

#[derive(Debug, Eq, PartialEq)]
pub enum PrepareError {
    UnrecognizedStatement,
    IncorrectParamNumber,
}
impl Display for PrepareError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            PrepareError::UnrecognizedStatement => {
                write!(f, "UnrecognizedStatement")
            }
            PrepareError::IncorrectParamNumber => {
                write!(f, "IncorrectParamNumber")
            }
        }
    }
}

#[derive(Debug, Eq, PartialEq)]
pub enum ExecuteError {
    UnrecognizedStatement,
    Failure(String),
}

impl Display for ExecuteError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            ExecuteError::UnrecognizedStatement => {
                write!(f, "UnrecognizedStatement")
            }
            ExecuteError::Failure(s) => {
                write!(f, "Failure: {}", s)
            }
        }
    }
}

pub enum ExecuteResult {
    Record(Vec<Row>),
    Affected(u32),
}
pub struct Statement {
    pub row_to_insert: Row,
    pub statement_type: StatementType,
}

#[derive(Serialize, Deserialize, Default, Debug, Clone)]
pub struct Row {
    pub id: u32,
    pub email: String,
    pub username: String,
}
impl Default for Statement {
    fn default() -> Self {
        Statement {
            statement_type: StatementType::Unrecognized,
            row_to_insert: Row::default(),
        }
    }
}

pub fn execute_statement(stmt: &Statement) -> Result<ExecuteResult, ExecuteError> {
    match stmt.statement_type {
        StatementType::Insert => execute_insert(stmt),
        StatementType::Select => execute_select(stmt),
        StatementType::Unrecognized => Err(ExecuteError::UnrecognizedStatement),
    }
}

fn execute_insert(_stmt: &Statement) -> Result<ExecuteResult, ExecuteError> {
    println!("This is where we would do an insert.");
    Err(ExecuteError::Failure("unimplemented".to_owned()))
}

fn execute_select(_stmt: &Statement) -> Result<ExecuteResult, ExecuteError> {
    println!("This is where we would do a select.");
    Err(ExecuteError::Failure("unimplemented".to_owned()))
}

This is the executor.rs file in the code, which contains functions for executing the SQL statements input by the user. The execute_statement function calls the corresponding function to execute the statement based on the type of input statement. If the statement type is "insert", the function calls the execute_insert function to perform the insert operation; if the statement type is "select", the function calls the execute_select function to perform the query operation; if the statement type is another value, the function returns ExecuteError::UnrecognizedStatement, indicating that the statement type is unrecognized.

The execute_insert and execute_select functions are both unimplemented placeholder functions that simply print a message and return an error.

The Row struct represents a row of data, containing the id, email, and username fields. The Statement struct represents a SQL statement, containing the type of statement to be executed and the row data to be inserted. The StatementType enum represents the types of SQL statements, including "insert" and "select". The PrepareError and ExecuteError enums represent possible errors that may occur when preparing and executing statements, including unrecognized statements and incorrect parameter counts.

In the implementation of the Display trait, we have implemented the fmt method for both PrepareError and ExecuteError to format the error messages.

Summary#

This article mainly introduces the basic flow and code implementation of REPL, including parsing user-inputted SQL statements and executing statements. The parser.rs file in the code contains functions for parsing user input and meta-commands, while the executor.rs file contains functions for executing SQL statements input by the user.