Memelang v11

De taal fungeert als een 'low-token' tussenrepresentatie (Intermediate Representation), die vervolgens programmatisch wordt vertaald naar de uitgebreidere SQL-code voor de database.

Voorbeeld van tokenreductie

Memelang (20 tokens): roles actor :$a="Mark Hamill";movie _;@ @ @;actor !$a

SQL (36 tokens): SELECT x.actor,x.movie,y.movie,y.actor FROM roles AS x,roles AS y WHERE x.actor='Mark Hamill' AND y.id!=x.id AND y.movie=x.movie

---

Technische Specificaties en Syntaxis

Memelang is ontworpen als een beknopte query DSL IR voor LLM text-to-SQL conversies.

Grammatica

De taal maakt gebruik van een axiale grammatica met de volgende hiërarchie: Axis2 → Axis1 → Axis0 → Cell

Hierbij zijn spaties syntactisch relevant en triggeren zij een "nieuwe Cell". Er mag geen spatie staan tussen operatoren, comparatoren, komma's of vlaggen en hun respectievelijke waarden.

Basis-syntaxis

De basisstructuur van de taal is als volgt gedefinieerd: [tabel WS] [kolom WS] [":$" variabele][":" ("min"|"max"|"cnt"|"sum"|"avg"|"last"|"grp")] [":" ("asc"|"des")] ["<=>" "\"" string "\""] [("="|"!="|">"|"<"|">="|"<="|"~"|"!~") (string|int|float|("$" variabele)|"@"|"_")] ";"

---

Gebruiksvoorbeelden

Hieronder volgen diverse scenario's waarin Memelang wordt toegepast om database-informatie op te vragen.

Eenvoudige selecties

• Alle films: movies ;;
• Alle rollen: roles ;;
• Titels en beschrijvingen van films: movies title ; description ;;
• Namen en leeftijden van acteurs: actors name ; age ;;

Filtering en condities

• Acteurs van 41 jaar of ouder: actors age >=41; _;;
• Specifieke rollen (ID 567 en 8901): roles id 567,8901; _;;
• Films met dystopische narratieven (sim > 0.33): movies description <=>"dystopian"<0.33; _;;
• Films met "Star" in de titel uit 1977 of 1980: movies title ~"Star"; year 1977,1980; _;;
• Acteurs met naam die lijkt op "Ana", leeftijd tussen 20 en 35 jaar: actors name ~"Ana"; age >=20;<=35; _;;

Complexe queries en joins

• Rollen met een score lager dan 1.5 voor films van vóór 1980:

movies year <1980; title ; roles movie @; rating <1.5; ;;

• Sortering op rating (dalend) en film (dalend):

roles rating :des; movie :des;;

• Alle films van vóór 1970, gesorteerd op jaar (oplopend):

movies year :asc<1970; _;;

• Gemiddelde performer-rating van minimaal 4.2:

roles rating :avg>=4.2; actor :grp;;

• Minimale rolbeoordeling per acteur, van laag naar hoog:

roles rating :min:asc; actor :grp;;

• Rollen in films waar "robot" in de beschrijving voorkomt en een rating van 3+ heeft:

movies description <=>"robot"<=$sim; title _; roles movie @; rating >=3;;

• Costars die gezien zijn met Bruce Willis of Uma Thurman:

roles actor :$a~"Bruce Willis","Uma Thurman"; movie _;@ @ @; actor !$a;;

Geavanceerde filters en limieten

• Oorlogsverhalen van vóór 1980: de top 12 films op basis van minimale rolbeoordeling:

movies year <1980; description <=>"war"<=$sim; title :grp; roles movie @; rating :min:des;%beg=0;%lim=12;;

• Rollen voor films "Hero" of "House of Flying Daggers" waar de acteur naam "Li" bevat, gesorteerd van A-Z:

movies title "Hero","House of Flying Daggers"; roles movie @; actor :asc~"Li";;

• Titels die "Here" bevatten over robots tussen 1900 en 2000:

movies title ~"Hero"; description <=>"robot"; year >=1900; <=2000;;

---

Referentie-implementatie (Python)

De onderstaande code vormt de basis voor de parsing en vertaling van Memelang naar SQL.

import re, sys, json
from typing import Optional, Union, List, Iterator, Pattern, Any
Err = SyntaxError

### SYNTAX ###
CELL_PATTERN = (
('QUO',   	r'"(?:[^"\\\n\r]|\\.)*"'),
('EMB',		r'\[(?:-?\d+(?:\.\d+)?)(?:\s*,\s*-?\d+(?:\.\d+)?)*\]'),
('MOD',		r'<->|<=>|<#>'),
('CMP', 	r'>=|<=|!~|!=|=|>|<|~|!'),
('BIND',	r':\$\w+'),
('FLAG',	r':[a-zA-Z]+'),
('VAR',		r'\$\w+'),
('REL',  	r'@\d?|\^'),
('WLD', 	r'_'),
('EVAR', 	r'%[a-zA-Z0-9_]+'),
('SLOT', 	r'#%?[a-zA-Z0-9_]+'),
('ASSN', 	r':#[a-zA-Z0-9_]+'),
('TIM',		r'\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}'),
('DEC',		r'-?\d*\.\d+'),
('INT',		r'-?\d+'),
('ALN',		r'[A-Za-z][A-Za-z0-9_]*'),
('OR',		r','),
('WS',		r'\s+'),
('MISMATCH', r'.'),
)
CANON = {'!':'!='}
CELL_REGEX=re.compile("|".join(f"(?P<{k}>{p})" for k, p in CELL_PATTERN))
PAD_MODES = {'qry','tab'}
FLAG_KINDS = {'FLAG','BIND','EVAR','ASSN'}
LIT_KINDS = {'TIM','DEC','INT','ALN','QUO','EMB'}
VAR_KINDS = {'VAR','WLD','REL','EVAR','SLOT'}
DAT_KINDS = LIT_KINDS | VAR_KINDS
RELCOORD = {
'@0': ['-1','-1'],
'@1': ['-1','-2'],
'@2': ['-1','-3'],
'@3': ['-1','-4'],
'@4': ['-1','-5'],
'@'  : ['-1','+0'],
'^'  : ['-1','end','+0'],
}

class Tok:
    def __init__(self, kind: str, src: str, canon: Optional[str] = None):
        self.kind = kind
        self.src = src
        canon = src if canon is None else canon
        self.canon = CANON.get(canon) or canon
        parser = {'QUO': json.loads, 'EMB': json.loads, 'DEC': float, 'INT': int}.get(kind)
        self.dat = parser(src) if parser else src
    def __str__(self): return self.src
    def __repr__(self): return self.canon
    def __eq__(self, other): return repr(self) == repr(other)
    def __hash__(self): return hash(self.src)
    def __bool__(self): return bool(self.src)

TOK_NULL = Tok('NULL', '')

class Seq(list[Tok]):
    opr: Tok = TOK_NULL
    def __init__(self, *items):
        super().__init__(items)
        self.opr = TOK_NULL
    def __str__(self): return self.opr.src.join([str(t) for t in self if len(str(t)) or t.kind=='HOLD'])
    def __repr__(self): return self.opr.src.join([repr(t) for t in self])

class Cell:
    flag: Seq
    left: Seq
    comp: Tok
    right: Seq
    padded = False
    def __init__(self, src: str):
        self.left = Seq()
        self.flag = Seq()
        self.comp = Tok('EQL', '', '=')
        self.right = Seq(Tok('WLD', '', '_'))
        toks = []
        for m in CELL_REGEX.finditer(src):
            kind = m.lastgroup
            text = m.group()
            if kind == 'WS': continue
            if kind == 'MISMATCH': raise Err(f'E_TOK {text!r}')
            toks.append(Tok(kind, text))
        i, n = 0, len(toks)
        def peek(): return toks[i].kind if i < n else ''
        def take():
            nonlocal i
            if i >= n: raise Err('E_EOF')
            t = toks[i]
            i += 1
            return t
        while peek() in FLAG_KINDS:
            self.flag.append(take())
        if peek() == 'MOD':
            self.left.opr = take()
            self.left.append(Tok('HOLD', ''))
            t = take()
            if not t.kind in DAT_KINDS: raise Err('E_TERM_DAT')
            self.left.append(t)
        if peek() == 'CMP':
            self.comp = take()
            if not peek() in DAT_KINDS: raise Err('E_DAT')
        if peek() in DAT_KINDS:
            self.right.clear()
            while peek() in DAT_KINDS:
                self.right.append(take())
            if peek() == 'OR':
                self.right.opr = take()
                if not peek() in DAT_KINDS: raise Err('E_OR_TRAIL')
        if i != n: raise Err(f'E_EXPR_TRAIL {toks[i:]}')

    def vectorize(self, tok: Tok) -> Tok:
        if tok.kind == 'EMB': return tok
        if tok.kind not in {'QUO', 'ALN'}: raise Err('E_EMBED')
        return Tok('EMB', json.dumps([0.1, 0.2]))

    @property
    def single(self) -> Tok:
        return self.right[0] if self.comp.canon == '=' and len(self.right) == 1 else TOK_NULL
    @property
    def literal(self) -> Tok:
        tok = self.single
        return tok if tok.kind in LIT_KINDS else TOK_NULL
    def find(self, kind:str) -> Tok:
        return next((flag for flag in self.flag if flag.kind == kind), TOK_NULL)
    def bind(self, tok: Tok):
        if tok not in self.flag: self.flag.append(tok)
    def __str__(self) -> str: return f"{self.flag}{self.left}{self.comp}{self.right}"
    def __repr__(self) -> str: return f"{self.flag!r}{self.left!r}{self.comp!r}{self.right!r}"
    def __bool__(self) -> bool: return bool(self.flag or self.left or self.right)

class Axis(list):
    src: str = ''
    sep: str = None
    sepreg: str = None
    sepstr: str = None
    empt: bool = False
    sub = None
    def __init__(self, src: str):
        if self.sep is None: raise Err('E_AXIS_SEP')
        if not self.sepreg: self.sepreg = re.escape(self.sep)
        if not self.sepstr: self.sepstr = self.sep + ' '
        self.src= src
        self.parse(src.strip())
    @property
    def regex(self) -> Pattern[str]:
        return re.compile(rf'''
(?P<COMM>"""(?:(?!""")[^\n\r\\]|\\.)*""")|
(?P<EXPQ>"(?:[^"\\\n\r]|\\.)*")|
(?P<SEP>{self.sepreg})|
(?P<EXPM>[^"{re.escape(self.sep[0])}]+)|
(?P<EXPS>.)
''', re.VERBOSE)
    def parse(self, src: str):
        exprs: List[str] = []
        for m in self.regex.finditer(src):
            if m.lastgroup == 'SEP':
                if exprs or self.empt:
                    self.append(self.sub("".join(exprs)))
                exprs.clear()
            elif m.lastgroup != 'COMM': exprs.append(m.group())
        if exprs: self.append(self.sub("".join(exprs)))
    def pull(self, coords):
        value = self
        for coord in coords: value = value[coord]
        return value
    def __str__(self) -> str:
        items = [str(t) for t in self]
        return self.sepstr.join([s for s in items if (s or self.empt)])

class Axis0(Axis):
    sep = ' '
    sepstr = ' '
    sepreg = r'\s+'
    sub = Cell

class Axis1(Axis):
    sep = ';'
    sub = Axis0

class Axis2(Axis):
    sep = ';;'
    sub = Axis1
    @staticmethod
    def coordrel(coords, rel):
        rel = ["+0"] * max(0, len(coords) - len(rel)) + [str(op) for op in rel]
        out = []
        for coord, op in zip(coords, rel):
            if op == "end": v=-1
            elif (v:=(coord + int(op)))<0: raise ValueError('E_REL_BIND')
            out.append(v)
        return out

    def rect(self):
        env = {'mode':'qry'}
        slots = [Tok('EVAR','','%tab'),  Tok('EVAR','','%col'), Tok('EVAR','','%val')]
        for idx2, axis1 in enumerate(self):
            idx = [idx2, None, None]
            for idx1, axis0 in enumerate(axis1):
                if not axis0: continue
                idx[1:] = [idx1, None]
                if bool(axis0[0].find('EVAR')):
                    for cell in axis0:
                        if not bool(cell.find('EVAR')): raise Err('E_AXIS_MET')
                        env[cell.find('EVAR').canon[1:]] = cell.single.dat
                    continue
                if axis0[0].single.kind=='SLOT':
                    slots=[]
                    for cell in axis0:
                        if cell.single.kind!='SLOT': raise Err('E_AXIS_SLOT')
                        if cell.single.canon[1]=='%': slots.append(Tok('EVAR', '', cell.single.canon[1:]))
                        else: slots.append(Tok('ASSN','',':'+cell.single.canon))
                    continue
                if env['mode'] not in PAD_MODES: continue
                axis0len=len(slots)
                if len(axis0) > axis0len: raise Err('E_AXIS0_LONG')
                for _ in range(axis0len - len(axis0)):
                    cell = axis0.sub('@')
                    cell.padded=True
                    axis0.insert(0, cell)
                for idx0, cell in enumerate(axis0):
                    idx[2] = idx0
                    cell.bind(slots[idx0])
                for seq in (cell.left, cell.right):
                    for n, tok in enumerate(seq):
                        if tok.kind!='REL': continue
                        coords=self.coordrel(idx, RELCOORD[tok.canon])
                        src=self.pull(coords)
                        if src.literal.kind != 'NULL':
                            seq[n] = src.literal
                            continue
                        name = '$'+'_'.join(map(str, coords)).replace('-1','E')
                        seq[n] = Tok('VAR', '', name)
                        src.bind(Tok('BIND', '', ':'+name))

PH = '%s'
Param = List[Union[int, float, str, list]]

class SQL:
    def __init__(self, sql: str = '', param: Optional[Param] = None):
        self.sql = sql
        self.param = [] if param is None else list(param)
    def sql_value(self) -> "SQL":
        return self
    def __str__(self) -> str:
        sql = self.sql
        for p in self.param: sql = sql.replace(PH, json.dumps(p), 1)
        return sql
    def __repr__(self) -> str: return str((self.sql, self.param))
    @staticmethod
    def uniq(terms: "SQL") -> list["SQL"]:
        out, seen = [], set()
        for term in terms:
            if term is None: continue
            key = (term.sql, tuple(map(repr, term.param)))
            if key in seen: continue
            seen.add(key)
            out.append(term)
        return out

class CellSQL(Cell):
    flag2agg = {':cnt':'COUNT', ':sum':'SUM', ':avg':'AVG', ':min':'MIN', ':max':'MAX', ':last':'MAX'}
    cmp2sql  = {'~':' ILIKE ', '!~':' NOT ILIKE '}
    mod2sql  = {}
    def __init__(self, src: str):
        super().__init__(src)
        self.base = self.alias = ''
        self.param = []
        flags = {t.canon for t in self.flag if t.kind == 'FLAG'}
        self.agg = next((sql for flag, sql in self.flag2agg.items() if flag in flags), '')
        self.grouped = ':grp' in flags
        self.sort = 'ASC' if ':asc' in flags else 'DESC' if ':des' in flags else ''
    def deref(self, bind: dict[str, SQL], with_agg: bool = True) -> Iterator[SQL]:
        for t in self.right:
            if t.kind == 'VAR':
                key = t.canon[1:]
                if key not in bind: raise Err(f'E_VAR_BIND {key}')
                ref = bind[key]
                if isinstance(ref, CellSQL): yield ref.sql_value(with_agg=with_agg)
                else: yield ref.sql_value()
            else:
                yield SQL(PH, [t.dat])
                continue
    @property
    def sql_groupby(self) -> Optional[SQL]:
        if not self.grouped: return None
        if self.agg: raise Err('E_GRP_AGG')
        return SQL(self.base, self.param)
    def sql_value(self, grouped: bool = False, alias: bool = False, order: bool = False, with_agg: bool = True) -> SQL:
        sql, param = self.base, list(self.param)
        if self.left.opr.kind == 'MOD':
            sql = f'({sql}{self.left.opr.canon}{PH}::VECTOR)'
            param.append(self.vectorize(self.left[1]).canon)
        agg = self.agg or ('MAX' if grouped and not self.grouped else '') if with_agg else ''
        if agg: sql = f'{agg}({sql})'
        if alias and self.alias: sql = f'{sql} AS {self.alias}'
        if order and self.sort: sql = f'{sql} {self.sort}'
        return SQL(sql, param)
    def sql_clause(self, bind: dict[str, SQL]) -> Optional[tuple[str, SQL]]:
        if not self.right or self.single.canon == '_': return None
        left = self.sql_value()
        rights = list(self.deref(bind, with_agg=bool(self.agg)))
        comp = self.comp.canon
        sqlcomp = self.cmp2sql.get(self.comp.canon) or self.comp.canon
        if comp in {'>', '<', '>=', '<='} and len(rights) != 1: raise Err('E_COMP_OR')
        items, params = [], []
        for right in rights:
            items.append(f"CONCAT('%', {right.sql}, '%')" if comp in {'~', '!~'} else right.sql)
            params.extend(right.param)
        if len(items) == 1: beg, end = '', ''
        elif comp in {'=', '~'}: beg, end = 'ANY(ARRAY[', '])'
        elif comp in {'!=', '!~'}: beg, end = 'ALL(ARRAY[', '])'
        else: raise Err('E_COMP_OR2')
        return ('having' if self.agg else 'where'), SQL(f"{left.sql}{sqlcomp}{beg}{','.join(items)}{end}", left.param + params)

class Grid(Axis2):
    def select(self) -> List[SQL]:
        self.rect()
        out = []
        env = {'mode':'qry', 'sim':0.5,'tab':'','taba':'','cola':''}
        for axis1 in self:
            env['lim'], env['beg'] = 0, 0
            bind = {k: SQL(PH, [v]) for k, v in env.items()}
            tab_cnt = 0
            qry = {'select':[], 'from':[], 'fromall':[], 'groupby':[], 'where':[], 'having':[], 'orderby':[]}
            grouped = False
            allselected = False
            for axis0 in axis1:
                if env['mode']!='qry': continue
                if axis0.src == '_':
                    allselected = True
                    continue
                for idx0, cell in enumerate(axis0):
                    single = cell.single.dat
                    if cell.padded or cell.single.kind=='SLOT': continue
                    evarval = cell.find('EVAR').canon
                    if evarval=='%val': pass
                    elif evarval:
                        env[evarval[1:]] = cell.single.dat
                        bind[evarval[1:]]=SQL(PH, [cell.single.dat])
                        if evarval=='%tab':
                            if not re.fullmatch(r'[A-Za-z_][A-Za-z0-9_$]{0,62}', single): raise Err('E_TAB_NAME')
                            tab_cnt += 1
                            env['tab']=single
                            env['taba']=f"t{tab_cnt}"
                            qry['from'].append(SQL(f"{env['tab']} AS {env['taba']}"))
                            qry['fromall'].append(env['taba'])
                        elif evarval=='%col':
                            if single == '_': allselected = True
                            elif not re.fullmatch(r'[A-Za-z_]+[A-Za-z0-9_$]{0,62}', single): raise Err('E_COL_NAME')
                            env['cola'] = single
                            continue
                    assnval = cell.find('ASSN').canon
                    if assnval: env['cola'] = assnval[2:]
                    if not env['taba']: raise Err('E_TAB_REQ')
                    valcell = CellSQL(repr(cell))
                    valcell.base = f"{env['taba']}.{env['cola']}"
                    qry['select'].append(valcell)
                    if valcell.grouped:
                        grouped = True
                        qry['groupby'].append(valcell)
                    if valcell.sort: qry['orderby'].append(valcell)
                    clause = valcell.sql_clause(bind)
                    if clause:
                        key, term = clause
                        qry[key].append(term)
                    for flag in valcell.flag:
                        if flag.kind != 'BIND': continue
                        if flag.canon[2:] in env: raise Err('E_ENV_BIND')
                        bind[flag.canon[2:]] = valcell
            if not qry['from']:
                out.append(SQL())
                continue
            parts = (
                ('SELECT', ', ', [SQL(f"{a}.*") for a in qry['fromall']] if allselected else SQL.uniq(t.sql_value(grouped, True) for t in qry['select'])),
                ('FROM', ', ', qry['from']),
                ('WHERE', ' AND ', qry['where']),
                ('GROUP BY', ', ', SQL.uniq(t.sql_groupby for t in qry['groupby'])),
                ('HAVING', ' AND ', qry['having']),
                ('ORDER BY', ', ', SQL.uniq(t.sql_value(grouped, False, True) for t in qry['orderby'])),
            )
            sql, param = [], []
            for keyword, sep, terms in parts:
                if not terms: continue
                sql.append(f"{keyword} " + sep.join(t.sql for t in terms))
                for t in terms: param.extend(t.param)
            if env['lim']: sql.append(f"LIMIT {int(env['lim'])}")
            if env['beg']: sql.append(f"OFFSET {int(env['beg'])}")
            out.append(SQL(' '.join(sql), param))
        return out

---

Licentie en Juridische Informatie

Deze software is gratis te gebruiken voor ontwikkelings-, test- en educatieve doeleinden. Commerciële implementatie, redistributie of gebruik in productie vereist een aparte licentie.

©2026 HOLTWORK LLC U.S. Pat. 12,475,098 Contact: info@memelang.net