Converting Plugins from PHP to JS
This page is about a switch in the technology used for our plugins, a few years ago.
Why the switch from PHP to JS?
Reasons for switching
- PHP is a pretty large dependency. Since it is not widely used as a “local” scripting language, most users will not have PHP installed on their systems. Particularily on Windows, this make installation yet more difficult.
- The PHP backend is terribly slow on Windows, and unfortunately there seems to be no easy fix for this.
- Since the PHP backend relies on a separate process, configuration issues may occur, such as problems with multi-byte local characters being garbled.
- The PHP backend works well for the code generation, but PHP cannot easily be used for GUI (logic) scripting [#Three uses for scripting]. This would mean there are at least two different scripting languages to learn.
- In cases of very complex plugins, the overhead of starting several PHP backends at once is noticeable.
Reasons against switching
- The PHP backend is proven technology. Switching is likely to introduce new bugs.
- Switching will be a lot of work, particularily for converting existing plugins.
- PHP is nice in that it has templating (<? and ?>), which is not available in most other languages. On the other hand, in most plugins the non-PHP regions are very fragmented, which takes away much of this benefit, and may even make the code hard to read.
Alternatives
- QtScript
- An ECMAscript (Javascript) implementation. The language is very wide-spread with many resources available, although Javascript is mostly used in a web-browsing context.
- Available directly in Qt. No additional dependencies, minimal overhead.
- Threadable. This is somewhat important for the code-generation backend ([#Three uses for scripting]), since code-generation should not make the GUI slow, even for very complex plugins.
- Kross (with a choice of different languages available)
- Allows to select from different languages (including QtScript), though it seems wise to restrict ourselves to only one language in the interest of maintainability.
- Available directly in KDE. No additional dependencies (as long
as one of the standard interpreters is used), low overhead.
- QtScript in Kross available since KDE 4.1.something.
- Unfortunately not threadable.
- Can we at least have several scripts in the same language in parallel? I hope so.
- R
- Obviously no additional dependencies.
- Generating R code in R code using paste() and friends is rather cumbersome
- Only one R command can run at a time. So an R command to generate code will be blocked by running calculations.
Of course the choice between Kross with QtScript/KJsEmbed and using QtScript directly is more of an implementation detail, and could easily be revised at a later point of time. The more important choice to make at this point is: Javascript or some other language.
Three uses for scripting
To clarify what we’re talking about: There are three main uses for scripting:
Code generation
Take settings in a plugin GUI, and convert them into R syntax. This is what the current PHP backend does. It’s also the only thing implemented for QtScript at this time.
GUI logic
Simple changes within in the GUI that happen in reponse to user input. This will be an alternative (not replacement!) for the current <logic>-section, which quickly becomes hard to manage for anything beyond very simple cases.
Technical note: In contrast to code generation, GUI logic needs to happen inside the same thread, i.e. in real-time. That’s the main reason why the code generation backend does not yet allow this.
Fully customized GUI elements
The elements provided in the xml files cover a whole lot of functionality, but in some cases it would be nice to be able to script specialized widgets for some tasks. Perhaps an advanced color picker. Or a small table to enter values for a chisq-test.
In theory a whole plugin could be entirely scripted instead of using an .xml-file. However - also in the interest of providing a consistent user interface - probably the real use case is for some small additions inside a plugin.
QtScript
What is there already?
Only the first scripting scenario is implemented so far, i.e. the code generation for which we used PHP so far. All PHP files have been converted to JS files by means of an evil little script. However, each single .js-file will have to be checked for correctness, manually, and small changes will be needed in the majority of files. See #Migration Plan.
What needs to be done short term?
Ideally, every plugin developer should pick one or two or their plugins, take a look at the generated .js file, Make the required corrections, and test the plugin. Then we’ll have the last chance to talk about fundamental changes, before we actually start the migration.
What needs to be done in the longer term?
- Each single JS file will need to be checked, activated, tested. Once all plugins are converted, the PHP script engine will be removed.
- The documentation needs to be updated (the plugins HOWTO, this wiki, installation instructions, etc.).
- Of course, the other two uses of scripting need to be implemented.
Migration plan
Overall plan
The PHP and JS script engine can, and do coexist. This means we need not do everything at once, but rather proceed one plugin at a time. However, the intent to finish this migration rather quickly, ideally before the next release. So let’s get started!
Steps to migrating a single plugin
- Take a look at the .js-file that was generated form your plugin’s .php-file. At the top there may be some messages generated by the conversion script. Heed these warnings, or, if you don’t know, what they are all about, ask about them on the list. See also the following sections.
- Even if there are no messages, take the time to read through the whole .js-file, to make sure it looks correct, overall.
- Once satisfied, in the plugin’s .xml-file, change the
-tag to point to the .js-file, instead of the .php-file.
- Test your plugin! Run the automated test(s) associated with it, but also test manually. The automated tests just don’t cover all aspects for most plugins.
- Run
svn remove PHPFILE.php
- Commit.
Noteable differences between QtScript and PHP
Syntax
Syntax is fairly similar for PHP and JS. However, variables are not marked with a ‘$’ at the beginning. If you have associative arrays ( Array (“x” => “value”, “y” => “valueb”) ) you can’t use the ‘=>’ in constructing them. Rather use myvar = new Object (); myvar.x = “value”; myvar.y = “valueb”;
Variable scopes and declaration
In PHP, if you wanted to use global variables (variables shared across functions), you had to declare them global, at the start of each function. In JS, the rules are different: If a variable was first declared in the global scope, it is global to all functions. If in PHP you had
function preprocess () {
global $a;
$a = 2; // global
$b = 1; // local
}
In JS you write
let a;
function preprocess() {
let b;
a = 2; // global
b = 1; // local
}
Note that often times it is not strictly necessary to declare the variables with the “var” keyword before using. However, it is good style, and highly recommended to declare each and every global and local variable, explicitly.
The conversion script should have done all this for you, so you can see what it is supposed to look like. In some cases you may want to edit a bit, e.g.:
let a; let b; a = 1; b = 2;
can be simplified to:
let a = 1; let b = 2;
Also, in many plugins you’ll find
let undefined;
somewhere. Delete those lines (bug in the conversion script).
Output
There are no “\<?” and “?>” tags for sections that should are output. Rather, everything is generated using “echo”-statements. The conversion script generally does a good job at converting this for you, but it may be thrown of in some circumstances (but that shows as syntax errors). This can significantly change the look of the plugin file, but I feel in most cases it’s actually to the better.
In PHP you could write
echo ("x is $variable");
in JS this does not work. Use
echo ("x is " + variable");
instead. The conversion script should have given you a warning about this.
Note that in JS, strings are concatenated by “+”, instead of “.”.
RKWard specific
getRK_val ("x");
is no longer. Use
getValue ("x");
instead. Instead of
getRK ("x");
use
echo (getValue ("x"));
The conversion script should have taken care of that, automatically.
Hints and specific instructions
- “Control statement without braces, this is bad style”
- Well, it’s not always bad style. In fact, most of these are false alarms. But sometimes you will end up with
if (a) echo ("something special"); echo ("something common");
That’s hard to read. Change that to
if (a) { echo ("something special"); } echo ("something common");
or better yet, insert a line break.