extend sublime text 3's sql syntax - syntax-highlighting

Question

This SO question covers the same general problem I have ... except that its answer is not detailed enough for me to understand what I must be doing wrong.
I am wanting to have sql files highlight any "CREATE PROCEDURE" commands.
I have the following file called sql-extension.tmLanguage in Packages\User\:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>fileTypes</key>
<array>
<string>sql</string>
</array>
<key>name</key>
<string>XYZ</string>
<key>patterns</key>
<array>
<dict>
<key>captures</key>
<dict>
<key>1</key>
<dict>
<key>name</key>
<string>keyword.other.create.sql</string>
</dict>
<key>2</key>
<dict>
<key>name</key>
<string>keyword.other.sql</string>
</dict>
<key>5</key>
<dict>
<key>name</key>
<string>entity.name.function.sql</string>
</dict>
</dict>
<key>match</key>
<string>(?i:^\s*(create)\s+(procedure)\s+)(['"`]?)(\w+)\4</string>
<key>name</key>
<string>meta.create.sql</string>
</dict>
<dict>
<key>include</key>
<string>#SQL</string>
</dict>
</dict>
<key>scopeName</key>
<string>source.sql</string>
<key>uuid</key>
<string>158e3bda-c76d-439e-b8ea-cb640f0a911c</string>
</dict>
</plist>
I copied this information from the SQL.tmLanguage file in the SQL package for ST3. Its content is this:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>fileTypes</key>
<array>
<string>sql</string>
<string>ddl</string>
<string>dml</string>
</array>
<key>foldingStartMarker</key>
<string>\s*\(\s*$</string>
<key>foldingStopMarker</key>
<string>^\s*\)</string>
<key>keyEquivalent</key>
<string>^~S</string>
<key>name</key>
<string>SQL</string>
<key>patterns</key>
<array>
<dict>
<key>include</key>
<string>#comments</string>
</dict>
<dict>
<key>captures</key>
<dict>
<key>1</key>
<dict>
<key>name</key>
<string>keyword.other.create.sql</string>
</dict>
<key>2</key>
<dict>
<key>name</key>
<string>keyword.other.sql</string>
</dict>
<key>5</key>
<dict>
<key>name</key>
<string>entity.name.function.sql</string>
</dict>
</dict>
<key>match</key>
<string>(?i:^\s*(create)\s+(aggregate|conversion|database|domain|function|group|(unique\s+)?index|language|operator class|operator|rule|schema|sequence|table|tablespace|trigger|type|user|view)\s+)(['"`]?)(\w+)\4</string>
<key>name</key>
<string>meta.create.sql</string>
</dict>
<dict>
<key>captures</key>
<dict>
<key>1</key>
<dict>
<key>name</key>
<string>keyword.other.create.sql</string>
</dict>
<key>2</key>
<dict>
<key>name</key>
<string>keyword.other.sql</string>
</dict>
</dict>
<key>match</key>
<string>(?i:^\s*(drop)\s+(aggregate|conversion|database|domain|function|group|index|language|operator class|operator|rule|schema|sequence|table|tablespace|trigger|type|user|view))</string>
<key>name</key>
<string>meta.drop.sql</string>
</dict>
<dict>
<key>captures</key>
<dict>
<key>1</key>
<dict>
<key>name</key>
<string>keyword.other.create.sql</string>
</dict>
<key>2</key>
<dict>
<key>name</key>
<string>keyword.other.table.sql</string>
</dict>
<key>3</key>
<dict>
<key>name</key>
<string>entity.name.function.sql</string>
</dict>
<key>4</key>
<dict>
<key>name</key>
<string>keyword.other.cascade.sql</string>
</dict>
</dict>
<key>match</key>
<string>(?i:\s*(drop)\s+(table)\s+(\w+)(\s+cascade)?\b)</string>
<key>name</key>
<string>meta.drop.sql</string>
</dict>
<dict>
<key>captures</key>
<dict>
<key>1</key>
<dict>
<key>name</key>
<string>keyword.other.create.sql</string>
</dict>
<key>2</key>
<dict>
<key>name</key>
<string>keyword.other.table.sql</string>
</dict>
</dict>
<key>match</key>
<string>(?i:^\s*(alter)\s+(aggregate|conversion|database|domain|function|group|index|language|operator class|operator|rule|schema|sequence|table|tablespace|trigger|type|user|view)\s+)</string>
<key>name</key>
<string>meta.alter.sql</string>
</dict>
<dict>
<key>captures</key>
<dict>
<key>1</key>
<dict>
<key>name</key>
<string>storage.type.sql</string>
</dict>
<key>10</key>
<dict>
<key>name</key>
<string>constant.numeric.sql</string>
</dict>
<key>11</key>
<dict>
<key>name</key>
<string>storage.type.sql</string>
</dict>
<key>12</key>
<dict>
<key>name</key>
<string>storage.type.sql</string>
</dict>
<key>13</key>
<dict>
<key>name</key>
<string>storage.type.sql</string>
</dict>
<key>14</key>
<dict>
<key>name</key>
<string>constant.numeric.sql</string>
</dict>
<key>15</key>
<dict>
<key>name</key>
<string>storage.type.sql</string>
</dict>
<key>2</key>
<dict>
<key>name</key>
<string>storage.type.sql</string>
</dict>
<key>3</key>
<dict>
<key>name</key>
<string>constant.numeric.sql</string>
</dict>
<key>4</key>
<dict>
<key>name</key>
<string>storage.type.sql</string>
</dict>
<key>5</key>
<dict>
<key>name</key>
<string>constant.numeric.sql</string>
</dict>
<key>6</key>
<dict>
<key>name</key>
<string>storage.type.sql</string>
</dict>
<key>7</key>
<dict>
<key>name</key>
<string>constant.numeric.sql</string>
</dict>
<key>8</key>
<dict>
<key>name</key>
<string>constant.numeric.sql</string>
</dict>
<key>9</key>
<dict>
<key>name</key>
<string>storage.type.sql</string>
</dict>
</dict>
<key>match</key>
<string>(?xi)
# normal stuff, capture 1
\b(bigint|bigserial|bit|boolean|box|bytea|cidr|circle|date|double\sprecision|inet|int|integer|line|lseg|macaddr|money|oid|path|point|polygon|real|serial|smallint|sysdate|text)\b
# numeric suffix, capture 2 + 3i
|\b(bit\svarying|character\s(?:varying)?|tinyint|var\schar|float|interval)\((\d+)\)
# optional numeric suffix, capture 4 + 5i
|\b(char|number|varchar\d?)\b(?:\((\d+)\))?
# special case, capture 6 + 7i + 8i
|\b(numeric)\b(?:\((\d+),(\d+)\))?
# special case, captures 9, 10i, 11
|\b(times)(?:\((\d+)\))(\swithoutstimeszone\b)?
# special case, captures 12, 13, 14i, 15
|\b(timestamp)(?:(s)\((\d+)\)(\swithoutstimeszone\b)?)?
</string>
</dict>
<dict>
<key>match</key>
<string>(?i:\b((?:primary|foreign)\s+key|references|on\sdelete(\s+cascade)?|check|constraint)\b)</string>
<key>name</key>
<string>storage.modifier.sql</string>
</dict>
<dict>
<key>match</key>
<string>\b\d+\b</string>
<key>name</key>
<string>constant.numeric.sql</string>
</dict>
<dict>
<key>match</key>
<string>(?i:\b(select(\s+distinct)?|insert\s+(ignore\s+)?into|update|delete|from|set|where|group\sby|or|like|and|union(\s+all)?|having|order\sby|limit|(inner|cross)\s+join|straight_join|(left|right)(\s+outer)?\s+join|natural(\s+(left|right)(\s+outer)?)?\s+join)\b)</string>
<key>name</key>
<string>keyword.other.DML.sql</string>
</dict>
<dict>
<key>match</key>
<string>(?i:\b(on|((is\s+)?not\s+)?null)\b)</string>
<key>name</key>
<string>keyword.other.DDL.create.II.sql</string>
</dict>
<dict>
<key>match</key>
<string>(?i:\bvalues\b)</string>
<key>name</key>
<string>keyword.other.DML.II.sql</string>
</dict>
<dict>
<key>match</key>
<string>(?i:\b(begin(\s+work)?|start\s+transaction|commit(\s+work)?|rollback(\s+work)?)\b)</string>
<key>name</key>
<string>keyword.other.LUW.sql</string>
</dict>
<dict>
<key>match</key>
<string>(?i:\b(grant(\swith\sgrant\soption)?|revoke)\b)</string>
<key>name</key>
<string>keyword.other.authorization.sql</string>
</dict>
<dict>
<key>match</key>
<string>(?i:\bin\b)</string>
<key>name</key>
<string>keyword.other.data-integrity.sql</string>
</dict>
<dict>
<key>match</key>
<string>(?i:^\s*(comment\s+on\s+(table|column|aggregate|constraint|database|domain|function|index|operator|rule|schema|sequence|trigger|type|view))\s+.*?\s+(is)\s+)</string>
<key>name</key>
<string>keyword.other.object-comments.sql</string>
</dict>
<dict>
<key>match</key>
<string>(?i)\bAS\b</string>
<key>name</key>
<string>keyword.other.alias.sql</string>
</dict>
<dict>
<key>match</key>
<string>(?i)\b(DESC|ASC)\b</string>
<key>name</key>
<string>keyword.other.order.sql</string>
</dict>
<dict>
<key>match</key>
<string>\*</string>
<key>name</key>
<string>keyword.operator.star.sql</string>
</dict>
<dict>
<key>match</key>
<string>[!<>]?=|<>|<|></string>
<key>name</key>
<string>keyword.operator.comparison.sql</string>
</dict>
<dict>
<key>match</key>
<string>-|\+|/</string>
<key>name</key>
<string>keyword.operator.math.sql</string>
</dict>
<dict>
<key>match</key>
<string>\|\|</string>
<key>name</key>
<string>keyword.operator.concatenator.sql</string>
</dict>
<dict>
<key>comment</key>
<string>List of SQL99 built-in functions from http://www.oreilly.com/catalog/sqlnut/chapter/ch04.html</string>
<key>match</key>
<string>(?i)\b(CURRENT_(DATE|TIME(STAMP)?|USER)|(SESSION|SYSTEM)_USER)\b</string>
<key>name</key>
<string>support.function.scalar.sql</string>
</dict>
<dict>
<key>comment</key>
<string>List of SQL99 built-in functions from http://www.oreilly.com/catalog/sqlnut/chapter/ch04.html</string>
<key>match</key>
<string>(?i)\b(AVG|COUNT|MIN|MAX|SUM)(?=\s*\()</string>
<key>name</key>
<string>support.function.aggregate.sql</string>
</dict>
<dict>
<key>match</key>
<string>(?i)\b(CONCATENATE|CONVERT|LOWER|SUBSTRING|TRANSLATE|TRIM|UPPER)\b</string>
<key>name</key>
<string>support.function.string.sql</string>
</dict>
<dict>
<key>captures</key>
<dict>
<key>1</key>
<dict>
<key>name</key>
<string>constant.other.database-name.sql</string>
</dict>
<key>2</key>
<dict>
<key>name</key>
<string>constant.other.table-name.sql</string>
</dict>
</dict>
<key>match</key>
<string>\b(\w+?)\.(\w+)\b</string>
<!-- <string>(\w+?)\.(\w+)</string> -->
</dict>
<dict>
<key>include</key>
<string>#strings</string>
</dict>
<dict>
<key>include</key>
<string>#regexps</string>
</dict>
</array>
<key>repository</key>
<dict>
<key>comments</key>
<dict>
<key>patterns</key>
<array>
<dict>
<key>captures</key>
<dict>
<key>1</key>
<dict>
<key>name</key>
<string>punctuation.definition.comment.sql</string>
</dict>
</dict>
<key>match</key>
<string>(--).*$\n?</string>
<key>name</key>
<string>comment.line.double-dash.sql</string>
</dict>
<dict>
<key>captures</key>
<dict>
<key>1</key>
<dict>
<key>name</key>
<string>punctuation.definition.comment.sql</string>
</dict>
</dict>
<key>match</key>
<string>(#).*$\n?</string>
<key>name</key>
<string>comment.line.number-sign.sql</string>
</dict>
<dict>
<key>begin</key>
<string>/\*</string>
<key>captures</key>
<dict>
<key>0</key>
<dict>
<key>name</key>
<string>punctuation.definition.comment.sql</string>
</dict>
</dict>
<key>end</key>
<string>\*/</string>
<key>name</key>
<string>comment.block.c</string>
</dict>
</array>
</dict>
<key>regexps</key>
<dict>
<key>patterns</key>
<array>
<dict>
<key>begin</key>
<string>/(?=\S.*/)</string>
<key>beginCaptures</key>
<dict>
<key>0</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.begin.sql</string>
</dict>
</dict>
<key>end</key>
<string>/</string>
<key>endCaptures</key>
<dict>
<key>0</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.end.sql</string>
</dict>
</dict>
<key>name</key>
<string>string.regexp.sql</string>
<key>patterns</key>
<array>
<dict>
<key>include</key>
<string>#string_interpolation</string>
</dict>
<dict>
<key>match</key>
<string>\\/</string>
<key>name</key>
<string>constant.character.escape.slash.sql</string>
</dict>
</array>
</dict>
<dict>
<key>begin</key>
<string>%r\{</string>
<key>beginCaptures</key>
<dict>
<key>0</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.begin.sql</string>
</dict>
</dict>
<key>comment</key>
<string>We should probably handle nested bracket pairs!?! -- Allan</string>
<key>end</key>
<string>\}</string>
<key>endCaptures</key>
<dict>
<key>0</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.end.sql</string>
</dict>
</dict>
<key>name</key>
<string>string.regexp.modr.sql</string>
<key>patterns</key>
<array>
<dict>
<key>include</key>
<string>#string_interpolation</string>
</dict>
</array>
</dict>
</array>
</dict>
<key>string_escape</key>
<dict>
<key>match</key>
<string>\\.</string>
<key>name</key>
<string>constant.character.escape.sql</string>
</dict>
<key>string_interpolation</key>
<dict>
<key>captures</key>
<dict>
<key>1</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.end.sql</string>
</dict>
</dict>
<key>match</key>
<string>(#\{)([^\}]*)(\})</string>
<key>name</key>
<string>string.interpolated.sql</string>
</dict>
<key>strings</key>
<dict>
<key>patterns</key>
<array>
<dict>
<key>captures</key>
<dict>
<key>1</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.begin.sql</string>
</dict>
<key>3</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.end.sql</string>
</dict>
</dict>
<key>comment</key>
<string>this is faster than the next begin/end rule since sub-pattern will match till end-of-line and SQL files tend to have very long lines.</string>
<key>match</key>
<string>(')[^'\\]*(')</string>
<key>name</key>
<string>string.quoted.single.sql</string>
</dict>
<dict>
<key>begin</key>
<string>'</string>
<key>beginCaptures</key>
<dict>
<key>0</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.begin.sql</string>
</dict>
</dict>
<key>end</key>
<string>'</string>
<key>endCaptures</key>
<dict>
<key>0</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.end.sql</string>
</dict>
</dict>
<key>name</key>
<string>string.quoted.single.sql</string>
<key>patterns</key>
<array>
<dict>
<key>include</key>
<string>#string_escape</string>
</dict>
</array>
</dict>
<dict>
<key>captures</key>
<dict>
<key>1</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.begin.sql</string>
</dict>
<key>3</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.end.sql</string>
</dict>
</dict>
<key>comment</key>
<string>this is faster than the next begin/end rule since sub-pattern will match till end-of-line and SQL files tend to have very long lines.</string>
<key>match</key>
<string>(`)[^`\\]*(`)</string>
<key>name</key>
<string>string.quoted.other.backtick.sql</string>
</dict>
<dict>
<key>begin</key>
<string>`</string>
<key>beginCaptures</key>
<dict>
<key>0</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.begin.sql</string>
</dict>
</dict>
<key>end</key>
<string>`</string>
<key>endCaptures</key>
<dict>
<key>0</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.end.sql</string>
</dict>
</dict>
<key>name</key>
<string>string.quoted.other.backtick.sql</string>
<key>patterns</key>
<array>
<dict>
<key>include</key>
<string>#string_escape</string>
</dict>
</array>
</dict>
<dict>
<key>captures</key>
<dict>
<key>1</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.begin.sql</string>
</dict>
<key>3</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.end.sql</string>
</dict>
</dict>
<key>comment</key>
<string>this is faster than the next begin/end rule since sub-pattern will match till end-of-line and SQL files tend to have very long lines.</string>
<key>match</key>
<string>(")[^"#]*(")</string>
<key>name</key>
<string>string.quoted.double.sql</string>
</dict>
<dict>
<key>begin</key>
<string>"</string>
<key>beginCaptures</key>
<dict>
<key>0</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.begin.sql</string>
</dict>
</dict>
<key>end</key>
<string>"</string>
<key>endCaptures</key>
<dict>
<key>0</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.end.sql</string>
</dict>
</dict>
<key>name</key>
<string>string.quoted.double.sql</string>
<key>patterns</key>
<array>
<dict>
<key>include</key>
<string>#string_interpolation</string>
</dict>
</array>
</dict>
<dict>
<key>begin</key>
<string>%\{</string>
<key>beginCaptures</key>
<dict>
<key>0</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.begin.sql</string>
</dict>
</dict>
<key>end</key>
<string>\}</string>
<key>endCaptures</key>
<dict>
<key>0</key>
<dict>
<key>name</key>
<string>punctuation.definition.string.end.sql</string>
</dict>
</dict>
<key>name</key>
<string>string.other.quoted.brackets.sql</string>
<key>patterns</key>
<array>
<dict>
<key>include</key>
<string>#string_interpolation</string>
</dict>
</array>
</dict>
</array>
</dict>
</dict>
<key>scopeName</key>
<string>source.sql</string>
<key>uuid</key>
<string>C49120AC-6ECC-11D9-ACC8-000D93589AF6</string>
</dict>
</plist>
I got the uuid from typing the following in the console:
import uuid
u = uuid.uuid4()
and then copy/pasting the UUID created into the file.
After saving the file and viewing a sql file, I get this (notice that the words "CREATE PROCEDURE" are not highlighted):
So what am I doing wrong?
EXTRA NOTE:
the AAAPackageDev package does not seem to work in ST3. All the menu options are greyed out.

Answer 1

For the following, I'm guessing you are using Windows, but if you aren't let me know (or find the command yourself). First, for the sake of demonstration, place the cursor over TABLE. Then press ctrl+alt+shift+p. In the status bar you will notice some text appear (something like source.sql meta.create.sql keyword.other.sql). This is the scope being applied to that word. The scope, along with your tmTheme file, define how it will be colored. Now place the cursor over PROCEDURE and repeat. You should see something similar to the scopes previously seen. If not, there is likely something wrong with your regular expression.
Oh, I would also add SQL to your ignored_packages setting if you haven't already. This will ensure that your syntax file is being used, not the built in one. You will probably want to copy out the contents of that package too as they probably provide some additional functionality.
AAAPackageDev is not compatible with ST3. ST3 uses Python3, whereas ST2 used Python2. So I wouldn't blindly expect a plugin to be compatible with both. Anyways, I'll try to provide some guidance on syntax highlighting (I'm no expert though). The first thing worth checking is that your regular expressions are correct. That is, you need to ensure the proper scopes are being applied to CREATE PROCEDURE.
In case you like to work in JSON rather than XML, take a look at PlistJsonConverter. It is compatible with ST3, though you will need to use the ST3 branch of it.
Edit
Leads me to believe there is something wrong with your regex. Is there any reason you don't just change
<string>(?i:^\s*(create)\s+(aggregate|conversion|database|domain|function|group|(unique\s+)?index|language|operator class|operator|rule|schema|sequence|table|tablespace|trigger|type|user|view)\s+)(['"`]?)(\w+)\4</string>
to
<string>(?i:^\s*(create)\s+(aggregate|conversion|database|domain|function|group|(unique\s+)?index|language|operator class|operator|rule|schema|sequence|table|tablespace|trigger|type|user|view)\s+)(['"`]?)(\w+)\4</string>
I simply added procedure to an existing regex pattern.