Thursday, August 25, 2016

Oomph 03: Our First Project Setup

We are going to prepare our first project setup with Oomph.

Source code for this tutorial is available on github as a single zip archive, as a Team Project Set or you can browse the files online.  

For a list of all Oomph related tutorials see my Oomph Tutorials Overview.

Step 1: A basic setup file

Of course Oomph setups can be created within eclipse. So start with a new General/Project named com.codeandme.oomph. Now create a new Oomph / Setup Project Model. The wizard will setup all kinds of default tasks for us, but we want to start from scratch to understand all the tasks involved.

So select a Simple Project, provide a Label "Code and me" and Finish the wizard.


The setup file created simply contains a root node for the project and a default stream named Master.


Not much of a setup yet, but we can already add it to our eclipse installer.

Step 2: Adding the setup to the installer

Launch the eclipse installer from our first tutorial, select an eclipse product and advance to page 2 of the wizard. There hit the + icon on the toolbar, select Browse File System and select the setup file we just created. The setup will be added to the <User> node. Not much sense installing our empty setup yet, but now the installer references our setup file and we can test it for each step we add. On each start of the installer the setup file gets reloaded, so you do not have to add the file each time you want to test it.

Step 3: Adding a simple ini file action

One of the simplest actions to add is an ini file adjustment. As you might guess this adds entries to the eclipse.ini file provided with your product installation. Select the Code and me node in your setup, then use the context menu to add a New Child / Eclipse ini task. Now we are going to adjust the task using the Properties view.

To change the initial heap size of your Java VM set Option to -Xms, Value to 1024m and VM to true. The last parameter is needed for all Java VM properties. Set it to false when changing any eclipse.ini properties like -showsplash or similar.

Make sure to provide a unique ID and a usable Description. The IDs are important when we want to refer to a node later in our setup.


Step 4: Add folders to structure your settings

When we start adding multiple ini tasks we should cluster them. Therefore add a new Compound item to your root node and D&D your ini settings in there. Compounds add more structure to your setup, but have no effect on the installation process.

Optional: Investigate common tasks

The scheme for adding new tasks is the same for all kinds of things. Add a node, adapt its properties, save and run your setup to test it. Good sources for tasks are the existing eclipse project setups. You may browse them by opening the catalog from the toolbar or from the main menu: Navigate / Open Setup / Parent Models / Catalog Index. We will have a closer look at dedicated tasks in the following tutorials.

Thursday, August 4, 2016

Regular Expression Tester

There are quite a lot regular expression tester utilities available for Eclipse. While all of them do a good job my favorite is RegExTester.

This project has not seen any updates for more than 2 years now, so some of you might consider it discontinued, I call it rock stable!

While working on my Oomph tutorials I found it really annoying that this little tool does not come with a p2 update site, so I created one and put it online. Feel free to use it for your own installations.

P2 site:

https://dl.bintray.com/pontesegger/regextester/

Wednesday, July 27, 2016

Oomph 02: A setup in action

During our first tutorial we started an installation using the Oomph installer. Now we will have a closer look on the applied tasks, how to monitor and relaunch them and where these settings get persisted.

Oomph Tutorials

For a list of all Oomph related tutorials see my Oomph Tutorials Overview.

Workspace Setup

Right after the installation Oomph prepares your workspace. While busy you can see a spinning icon in the status bar at the bottom of your eclipse installation. A double click reveals a progress dialog where you can investigate all actions Oomph performs.
Oomph provides a toolbar, which is hidden by default. Enable it in Preferences / Oomph / Setup Tasks by checking Show tool bar contributions. Now we can repeat setup tasks or add additional project setups to our installation using the Import Projects... wizard from the toolbar setup entry.

Preferences Recorder

One of the most interesting features of Oomph is the preferences recorder. It can be enabled in the preferences window by selecting the record item in the bottom left corner. Once enabled it records all preference changes and stores them for you. When switching to another workspace these settings are applied directly. In practice this means: change a setting once and as long as you stick to Oomph you never have to think about it anymore.

Generally setup tasks (like setting preferences) may be stored to one of three different locations:
  1. User
    This is a global storage on your local machine shared for all installations and workspaces. Most of your changes will go here.
  2. Installation
    Settings get stored in the configuration folder of your current eclipse installation. These settings apply as long as you stick to the current eclipse binary.
  3. Workspace
    These settings get stored in the .metadata folder of your current workspace. So they are workspace specific, no matter which eclipse binary you use to access this workspace.
Personally I did not find a use case for options 2 or 3 yet.

Investigate Oomph Setups

Now that we know of the different storage locations, we can have a look at their content. The second toolbar item allows to open each one of them in the Setup Editor (setups are also available from the Navigate Open Setup menu).

The editor displays a tree structure of all Oomph tasks. As it is based on EMF we have to open the Properties view to display details of each tree element. If an element has a [restricted] annotation next to its name this means that the definition of this item is referenced by the current setup file. Typically this refers to a setup stored on the web. Such entries are readable, but cannot be changed without opening the original setup file.

Now that you are familiar with the basic ingredients we are ready to start building our own project setups.

Thursday, July 21, 2016

Oomph 01: A look at the eclipse installer

This will be the start of a new series of posts on Oomph. It is the basis for the eclipse installer but with the right configuration it can do much more:
  • serve your own RCP applications
  • provide fully configured development environments
  • store and distribute your settings over multiple installations
to name a few. This first part will have a look at the installer itself. Further posts of this series will focus on custom setups and how we can leverage the power of Oomph.

Oomph Tutorials

For a list of all Oomph related tutorials see my Oomph Tutorials Overview.

Step 1: The Eclipse Installer

All starts with the Eclipse Installer, a tool we will need throughout this series. Download and unzip it. The installer is an eclipse application by itself, so it provides the familiar folder structure with plugins, features, ini file and one executable. As we will need the installer continuously find a safe home for it.

After startup you will be in Simple Mode, something we will not cover here. Use the configuration icon in the top right corner to switch to Advanced Mode. The first thing we are presented with is a catalog of products to install.
The top right menu bar allows to add our own catalogs and to select which catalogs are displayed. For now we will ignore these settings, they will be treated in a separate tutorial. After selecting a product the bottom section allows us to select from different product versions, 32/64 bit, the Java runtime to use and if we want to use bundle pools.


Bundle Pools

A bundle pool is a place that stores - among some other things - plugins and features. Basically everything that a typical eclipse application would host in its plugins/features folders. Further it may host the content of target platforms.

Using a shared bundle pool saves you from a lot of redundant downloads from eclipse servers and can provide offline abilities. For everything available in the bundle pool you do not require an internet connection anymore. A nice feature if you are sitting behind a company firewall. While it is not required to use them, bundle pools save you a lot of time and are safe and convenient to use. At first I was quite hesitant of splitting my installations and move stuff to bundle pools, but after giving it a try I do not want to step back anmore.

To have some control over the used bundle pools, click on the icon next to the location and setup a New Agent... on a dedicated location. Further eclipse installations will use this pool, so do not alter the directory content manually. The Bundle Pool Management view will allow you to analyze, cleanup and even repair the directory content.
Step 2: Project Selection

The 2nd page of the installer presents eclipse projects we want to add to our installation. Selecting a project typically triggers actions after the plain eclipse installation:
  • automatically checkout projects
  • import into the workspace
  • set the target platform
  • apply preference settings
  • setup Mylyn
  • install additional components
The target is that you get everything pre-configured to start coding on the selected projects.

Step 3: Installer Variables

Installations do need some user input for the install location, repository checkout preferences, credentials and more. All these accumulated variables will be presented on the next page of the installer. By default the installer creates three folders below the Root install folder:
  • eclipse
    to host the eclipse binary and configuration. If you use bundle pools plugins and features go there. Otherwise they will be located here.
  • ws
    the default workspace for this installation
  • git
    the default folder for git checkouts
You may go with these defaults or change them to your needs. While developing a setup (which we will start in an upcoming tutorial) I would recommend to use these settings. For a final installation I prefer to host my workspace elsewhere.

Oomph stores all your settings in a global profile. So the next time you install something it will use your previously entered values here. You may always revisit your choices by enabling Show all variables in the bottom left corner.

The last page finally allows you to enable offline mode and to enable/disable download mirrors. On the next tutorial we will have a closer look at setup tasks and where these settings get persisted.

Optional: Preferences

The icons on the bottom allow to set two kinds of important preferences: proxy and ssh settings. If you are behind a proxy activate those settings and they will automatically be copied to any installation done be Oomph.

Ssh might be needed for git checkouts depending on your repository choices. If you do not use the default ssh settings you might need to wait for Neon.1 to have these settings applied (see bug 497057).

Thursday, May 12, 2016

EASE release, Community award & EclipseCon France

EASE v0.3.0 release

Today we shipped the brand new v0.3.0 release of EASE, your beloved scripting framework. Most important we improved the installation process for Jython as you do not need to know about external update sites anymore.

Code completion got introduced to help you browse your script objects and to improve coding speed. Completion proposals are provided for module functions, Java classes and even for some parameters (like file paths). In case you run eclipse from a JDK we also provide help popups for java methods.



Then we added a whole bunch of new commands to our modules to simplify your scripts. The Scripting module now provides means to exchange data between multiple script instances, the UI module got new methods to manipulate views and the brand new P2 module allows to install or update packages.

Of course we also had lots of bugfixes, if you are interested you may browse the complete list of changes.

Community Award

While the release might be the most important news for you, the development team is really proud of the Eclipse Community Award for the Most Innovative Project 2016.


It is a real honor that you, the community, rate this little project so high. While there seems to be quite some userbase using EASE, I have little knowledge of the adopters out there. I would love to read who you are and what you are using EASE for. If you got a minute left, please leave a comment to this blog entry.

EclipseCon France

In case you have no idea why this project might be innovative, meet us at EclipseCon France, where there will be several talks on EASE. I will be involved in 2 of them:

On Tuesday afternoon there will be a tutorial session EASE-ily Make the Most of Eclipse with Python, where we will do a lot of scripting (of course in Python). Besides scripts we will also look on how to integrate such scripts into the IDE and how to distribute them successfully.

On Thursday morning I will be giving a talk on the EASE framework itself, showing how to Elevate your IDE with scripts. We will see what scripting is capable of and I will give a sneak preview of the upcoming features for the 0.4 release. I will just say so much: your scripts never integrated so well and easily with your IDE.

While those talks are already worth going to EclipseCon, there is also tons of other interesting topics to investigate, so be there!

Monday, April 25, 2016

A new interpreter for EASE (5): Support for script keywords

EASE scripts registered in the preferences support  a very cool feature: keyword support in script headers. While this does not sound extremely awesome, it allows to bind scripts to the UI and will allow for more fancy stuff in the near future. Today we will add support for keyword detection in registered BeanShell scripts.

Read all tutorials from this series.

Source code for this tutorial is available on github as a single zip archive, as a Team Project Set or you can browse the files online. 

Step 1: Provide a code parser

Code parser is a big word. Currently all we need to detect in given script code are comments. As there already exists a corresponding base class, all we need to do is to provide a derived class indicating comment tokens:
package org.eclipse.ease.lang.beanshell;

import org.eclipse.ease.AbstractCodeParser;

public class BeanShellCodeParser extends AbstractCodeParser {

 @Override
 protected boolean hasBlockComment() {
  return true;
 }

 @Override
 protected String getBlockCommentEndToken() {
  return "*/";
 }

 @Override
 protected String getBlockCommentStartToken() {
  return "/*";
 }

 @Override
 protected String getLineCommentToken() {
  return "//";
 }
}

Step 2: Register the code parser

Similar to registering the code factory, we also need to register the code parser. Open the plugin.xml and select the scriptType extension for BeanShell. There register the code parser from above. Now EASE is able to parse script headers for keywords and interprets them accordingly.

Wednesday, December 16, 2015

A new interpreter for EASE (4): Provide modules support

EASE is shipped with modules: libraries written in Java, made available for script interpreters. The intention of such modules is to write them once and use them in all available script languages.

Read all tutorials from this series.

Source code for this tutorial is available on github as a single zip archive, as a Team Project Set or you can browse the files online. 


Introduction

Typically modules are plain java classes. We could easily create an instance in BeanShell and then call instance.method(). While this is perfectly ok for the average programmer, some of our user might expect a more simple, functional interface. What module loading does is:
  • create an instance of the module
  • inject the instance into the interpreter
  • create dynamic script code to wrap method calls
  • execute the created code
For BeanShell we would create code like
method(param1, param2) { 

result = __MOD_module_instance.method(param1, param2);

return result;

}
After loading this fragment we could access method() without the knowledge of modules, instances and object orientation at all.

Step 1: The code factory

To dynamically create code EASE needs an ICodeFactory implementation. To start you may look at existing implementations, basically we just need to build strings containing script code.
package org.eclipse.ease.lang.beanshell;

import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.util.Arrays;
import java.util.List;

import org.eclipse.ease.Logger;
import org.eclipse.ease.modules.AbstractCodeFactory;
import org.eclipse.ease.modules.IEnvironment;
import org.eclipse.ease.modules.IScriptFunctionModifier;
import org.eclipse.ease.modules.ModuleHelper;

public class BeanShellCodeFactory extends AbstractCodeFactory {

 public static final List<String> RESERVED_KEYWORDS = Arrays.asList("abstract", "continue", "for", "new", "switch", "assert", "default", "goto", "package",
   "synchronized", "boolean", "do", "if", "private", "this", "break", "double", "implements", "protected", "throw", "byte", "else", "import", "public",
   "throws", "case", "enum", "instanceof", "return", "transient", "catch", "extends", "int", "short", "try", "char", "final", "interface", "static",
   "void", "class", "finally", "long", "strictfp", "volatile", "const", "float", "native", "super", "while");

 private static boolean isValidMethodName(final String methodName) {
  return BeanShellHelper.isSaveName(methodName) && !RESERVED_KEYWORDS.contains(methodName);
 }

 @Override
 public String getSaveVariableName(final String variableName) {
  return BeanShellHelper.getSaveName(variableName);
 }

 @Override
 public String createFunctionWrapper(final IEnvironment environment, final String moduleVariable, final Method method) {

  final StringBuilder scriptCode = new StringBuilder();

  // parse parameters
  final List<Parameter> parameters = ModuleHelper.getParameters(method);

  // build parameter string
  final StringBuilder parameterList = new StringBuilder();
  for (final Parameter parameter : parameters)
   parameterList.append(", ").append(parameter.getName());

  if (parameterList.length() > 2)
   parameterList.delete(0, 2);

  final StringBuilder body = new StringBuilder();

  // insert hooked pre execution code
  body.append(getPreExecutionCode(environment, method));

  // insert deprecation warnings
  if (ModuleHelper.isDeprecated(method))
   body.append("\tprintError('" + method.getName() + "() is deprecated. Consider updating your code.');\n");

  // insert method call
  body.append("\t ");
  if (!method.getReturnType().equals(Void.TYPE))
   body.append(IScriptFunctionModifier.RESULT_NAME).append(" = ");

  body.append(moduleVariable).append('.').append(method.getName()).append('(');
  body.append(parameterList);
  body.append(");\n");

  // insert hooked post execution code
  body.append(getPostExecutionCode(environment, method));

  // insert return statement
  if (!method.getReturnType().equals(Void.TYPE))
   body.append("\treturn ").append(IScriptFunctionModifier.RESULT_NAME).append(";\n");

  // build function declarations
  for (final String name : getMethodNames(method)) {
   if (!isValidMethodName(name)) {
    Logger.error(IPluginConstants.PLUGIN_ID,
      "The method name \"" + name + "\" from the module \"" + moduleVariable + "\" can not be wrapped because it's name is reserved");

   } else if (!name.isEmpty()) {
    scriptCode.append(name).append("(").append(parameterList).append(") {\n");
    scriptCode.append(body);
    scriptCode.append("}\n");
   }
  }

  return scriptCode.toString();
 }

 @Override
 public String classInstantiation(final Class<?> clazz, final String[] parameters) {
  final StringBuilder code = new StringBuilder();
  code.append("new ").append(clazz.getName()).append("(");

  if (parameters != null) {
   for (final String parameter : parameters)
    code.append(parameter).append(", ");

   if (parameters.length > 0)
    code.delete(code.length() - 2, code.length());
  }

  code.append(")");

  return code.toString();
 }

 @Override
 public String createFinalFieldWrapper(final IEnvironment environment, final String moduleVariable, final Field field) {
  return getSaveVariableName(field.getName()) + " = " + moduleVariable + "." + field.getName() + ";\n";
 }

 @Override
 protected String getNullString() {
  return "null";
 }
}
createFunctionWrapper() is called for each exposed method, createFinalFieldWrapper() is called for each exposed final field. The function wrapper injects preExecutionCode and postExecutionCode. This is interesting for modules implementing IScriptFunctionModifier. If such a module is loaded, it may influence code generation for all other modules. Use cases are creation of log files or unit testing.

Step 2: Register code factory

Now we need to register our code factory, so EASE is able to find it. Open the Extensions tab of org.eclipse.ease.lang.beanshell/plugin.xml. Navigate to the scriptType extension and add the code factory class to the BeanShell type.
Step 3: Bootstrapping

There exists a dedicated Environment module that provides basic commands like loadModule() to load further modules. Lets try to load it in a BeanShell:

run EASE and open a bean shell in the Script Shell View. We will create an instance of EnvironmentModule and advise it to load itself:
new org.eclipse.ease.modules.EnvironmentModule().loadModule("/System/Environment");
If your code factory works as expected, commands like print() and loadModule() should be available.

To automate this task we will add a launch extension to our BeanShell language definition.
Switch to the plugin.xml and add a launchExtension to the org.eclipse.ease.language node.We bind it to the engineID of BeanShell and need to provide an implementation:
public class BootStrapper implements IScriptEngineLaunchExtension {

 @Override
 public void createEngine(final IScriptEngine engine) {
  ICodeFactory codeFactory = ScriptService.getCodeFactory(engine);
  if (codeFactory != null) {
   StringBuilder stringBuilder = new StringBuilder();
   stringBuilder.append(codeFactory.classInstantiation(EnvironmentModule.class, new String[0]));
   stringBuilder.append(".loadModule(\"");
   stringBuilder.append(EnvironmentModule.MODULE_NAME);
   stringBuilder.append("\");\n");

   engine.executeAsync(new Script("Bootloader", stringBuilder.toString()));
  }
 }
}
Of course we could directly use the string we tried manually before, but using the code factory is somewhat nicer. The implementation given above is already available from the BootStrapper class from EASE. If you do want to add more stuff to the bootloader you need to provide your own implementation.

With that in place you should be able to load all available modules into BeanShell and use its functions.

Optional: Loading external jars

We added support to register URLs to the classloader in our previous tutorial. Now as we have modules working we may try this out by calling
loadJar("http://central.maven.org/maven2/com/github/lalyos/jfiglet/0.0.7/jfiglet-0.0.7.jar")
 true
com.github.lalyos.jfiglet.FigletFont.convertOneLine("I      love      scripting");
  ___        _                                             _         _    _               
 |_ _|      | |  ___  __   __  ___        ___   ___  _ __ (_) _ __  | |_ (_) _ __    __ _ 
  | |       | | / _ \ \ \ / / / _ \      / __| / __|| '__|| || '_ \ | __|| || '_ \  / _` |
  | |       | || (_) | \ V / |  __/      \__ \| (__ | |   | || |_) || |_ | || | | || (_| |
 |___|      |_| \___/   \_/   \___|      |___/ \___||_|   |_|| .__/  \__||_||_| |_| \__, |
                                                             |_|                    |___/