Erlang Project Ideas

Probability of Project Success

Erlang Dev Environment

I have started a tutorial on how to setup a professional development environment for erlang coding.  The github page below has better formatting.

https://gist.github.com/andytill/6094931

South Park Software License

If you have ever watched the show South Park then you will have probably read the disclaimer that is flashed at the start.  Here is my software license tribute:


Remember that software development can often be much like an episode of South Park; constant swearing, youthful naivety and surprise endings.

Memories of Visual Basic 6

In the UK it is typical to study for two years and then have a placement year where you actually get a job before the final fourth year of study.  In 2006 I took on several small projects over the placement  year and pushed VB6 to it's limits.  VB6 has so many quirks, it is impossible not to use hacks to get things done.  Here are some of mine.


One tool required a DSL for interacting with phones, it would read a list of phone numbers and a file containing commands and run them over each of the numbers.  For example it would make a call to the phone and configure it using DTMF* tones.  I had no idea at that time how a compiler could be properly implemented so the resulting code was a complete mess and the DSL was looked like assembly for telephones.  Despite the hideous code, the program saved a lot of time.  To do this manually would have taken days instead of a couple of hours and be impossibly error prone.  One key missing feature was error handling, which is problematic in telecommunications.  Inevitably one of the calls would go wrong and the program would crash, the operator would then have to delete the completed phones numbers from the list and run the program again.  No problems!

Another project provided a UI for editing INI file configuration.  The program was great, except that VB does not allow more than 256 controls on a single form.  My 'ingenious' solution was to put each INI section in a tab pane which also had its own window.  When the user clicked a new tab, it hid the current window and showed a new window for that tab. 

VB6 also comes with it's own TCP control which you must drag onto the UI to instantiate.  The problem is that it had almost the same API as a serial connection control, meaning it only dealt with 1 to 1 connections.  This meant it was impossible to write a TCP server.  That is, until I worked out that an array of TCP controls waiting for connections worked pretty much like a TCP server.  This is OK except that an array has a finite capacity, I was again limited by the number of controls so the server could only ever handle a certain number of connections.

I remember the year, the work and the company fondly and hope that some of my work is still in use.  Looking back, I don't know why I used VB6 for everything.  It's just so easy to get started and then keep typing, I was young and naive, that's all I can say...

* DTMF tones are those strange sounds you hear when you press a number on a phone key pad.  Pieces of hardware such as phones and switches can interpret a sequence of tones as commands.

EstiMate, a software estimation tool

I have just open sourced a software application that I have been working on in my spare time for the past few months called EstiMate. This is my attempt to create some software which improves the way I can estimate a task and make what that estimate includes more transparent so that it is clear to others and Myself in the future. Areas which I feel are particularly important are:

• Task complexity, how complex it the task and how confident is the assignee with that technology.
• Quickly documenting what a task consists of and breaking it into smaller tasks
• Having a history of past estimates and how they worked out.

There are many many more factors which affect estimation but these are the things that I want to implement first.

So, what does it do? At the moment it has some characteristics of an ERP application. You are able to add some projects, tasks and people to implement them. Tasks can be estimated and the sum of estimates are calculated for a project.

Complexity is taken into account which I think is a huge factor in any task. Tasks can be assigned "skill requirements" such as Java, SQL or anything you think you need and then can be rated out of ten. People can assign skills to themselves and rate themselves. If a task has skill requirements greater than those possessed by the assigned person then the estimate for a task is increased. I plan to improve this calculation by allowing the estimate to be increased by a percentage set in the project, instead of by a fixed number. Other factors affecting the time that a task can take will also be added.

Tasks can also be "templated", this means that the information entered for this task can be recreated. This is great for reoccurring tasks like builds, deployments or coding tasks that are very similar. It will allow these tasks to look similar and have a consistent base for estimation but be flexible and allow changes as needed. I plan to improve this to capture best practices such as including sub-tasks for testing or bug fixing that are percentages of the task estimate or perhaps even complexity.

EstiMate uses JavaFX for the frontend GUI and neo4j for data storage. It allows the user to keep a local database, to use a shared neo4j server or a neo4j database hosted on the Heroku platform. It is pure java and shouldn't need any set-up once I create an installer. At the moment, you can download the code from bitbucket and run it from your favourite IDE or command line (there might be a Maven hack there right now so be warned!).

It is licensed under GPLv3. I have blogged about some of the code I have written for this project so assume any code in this blog is free for you to use in any way you see fit without recourse to Me if it doesn't work of course!

If you have an idea or opinion or would like to contribute that would be great, feel free to comment here or tweet me @andy_till.

Dragging to resize a JavaFX Region

Here is another handy JavaFX utility to allow you to drag the bottom edge of a JavaFX Region to resize it. Layout containers inherit from Region although controls do not even though they both have a minHeight property so Controls cannot be resized using this class :(

Features I would like to see in FXML

I really enjoy using FXML, it is robust, simple and well supported by the Scene Builder tool but there are some features I would really like to see.

Controllers allowed on arbitrary nodes

Large controls usually have many action methods, many controls injected with the @FXML annotation and typically other objects like DAOs which can get quite messy.  Sometimes handling code for a single control such as a combo box can get quite large, creating an FXML for a small control seems like overkill.

To tackle this, it would be useful to allow the fx:controller attribute on any FXML element specifying a node that is loaded into the scene graph.  This 'sub-controller' would have controls injected with @FXML annotations in the same way as root controller except that the it would not be able have it's parent controls injected.  Only the element it is declared on or that element's children could be injected.

Allow effects to be created in Scene Builder and then injected into a controller

It is much easier to create effects in Scene Builder than it is in Java code because I can get instant feedback and fine tune much easier.  It would be great if I could create an effect in FXML and have it injected into the controller using an @FXML annotation.

Support 'Actions'

One of the things that I miss from Swing is the javax.swing.Action interface.  This was used to encapsulate action logic of a button in a single class instead of having many action listeners in one class.  I could create a class implementing EventHandler and set it on the button in the controller class, but it would be nice to specify it under the code tab in the same as you would specify a method to handle an event.

Specify some validations on TextField controls

For Me, FXML is all about productivity so it makes sense that it could support common requirements such as validations on TextField controls.  For example, a TextField might only allow numeric input up to a maximum number.  If you google "javafx numeric textfield" there are already many varied implementations.  It would be great to see this in Scene Builder, even better if it could be tested in preview mode.

JUnit Rule for JavaFX Controller Testing

Testing JavaFX controllers is relatively easy although you do have to jump through a couple of hoops. Firstly, JavaFX must be initialised and secondly JavaFX controls must only be mutated on the JavaFX thread created internally during initialisation. The JUnit rule below will setup JavaFX once and run all tests on the FX thread. All you need to do is create it as a field in your test case class and annotate it with @org.junit.Rule. You will need to be running JUnit 4.9 to use the rule API.

Reusable FXML Controls with Guice

Update:
This code is now integrated into the fx-guice project where it will be maintained.  I am not intending to modify the GitHub gists so this post now only serves to document the code as seen here.

---

It is very simple to inject controllers into an FXML control using Guice but once the UI starts to get bigger it is likely that we will need to split it out into smaller files. I wanted to do this with a list builder control simple to that in SceneBuilder used to build a list of CSS classes.

In my UI the controller of the parent control (called parent controller from now on) would tell the list builder control what items it could add to the list and what items were already in the list using the list builder's controller (the child controller).  Getting this to work this is far from simple.  

It is not possible to get or set a controller on an arbitrary control node.  I fired up VisualVM to see where the list builder controller was referenced and the only incoming references were to the event handlers I had set up.  If there were no event handlers it looks like the controller would get garbage collected.  It was also not possible to set a Guice singleton scope on the controller as there may be several list builders in the user interface at the same time.

Here is a run down of my final solution although it is not altogether as simple as I would have liked it.  This tutorial is much more complex than my previous FXML/Guice post, I had to look up my own tutorial on Guice scopes to work it out.  Please let me know if anything is not clear

1. Add the Guice Module

Add the following Guice module to your Injector.  This example also assumes that you are using the same Injector to inject controllers when loading FXML, this is detailed on my previous Guice/FXML post.

The module uses the following Guice scope to track all injected controllers.

2. Use a scope annotation on the controller

Reusable controllers are annotated with the custom @FXMLLoadingScope.

3. Implement the IdentifiableController interface in the controller

Reusable controllers must implement the IdentifiableController interface which has one method, #getId().  #getId() must return a non-null String containing the ID of the parent of the root pane of the control.  If this ID is null then return the second parent-parent's ID.  Keep going until a non-null ID is found.

4. Set an ID on the parent

When importing one FXML file into a parent, set the container ID to the ID you want for the controller.

5. Retrieve the child controller

In our parent controller (the one that that will tell the list builder what to do) we need to retrieve the list builders controller to populate options and so on.  To do this, Inject an instance of ControllerLookup into the parent controller.
In the parent controller, make sure you implement the Initializable interface.  In the initialise method of the parent controller call ControllerLookup#lookup(String) on the instance you have injected to retrieve the child controller.  The ID that is passed in should be the ID that is returned by IdentifiableController#getId().  This could be hard coded but in this example it is using the FXML ID of the parent control, "factorsList" in this case.  This allows several list builders to be used in the same parent control.

I am using this in my own JavaFX project and although the implementation was not simple I have found it quite easy to use.

JavaFX CSS Styling

I've been working on a light grey look and feel for JavaFX with square corners, inspired by the Swing Substance (Business Grey) Look and Feel.  Here are the results so far:




And just as a reminder as to the default JavaFX look and feel:



This is a work in progress and not all of the controls are styled.  This skin has now become the first entrant in the jfxtras-styles project hosted on github or click here if you just want to grab source.

The extra space is to show the drop shadows on the titled panes which I use quite a lot in my own application.

Update:
Just to show the different a little CSS styling can do, below is a recent screenshot of EstiMate, the JavaFX project I have been working on.  It's come a long way!  You can take a look at the CSS here.

Creating JavaFX Controllers Using Guice

I really like Guice Dependency Injection so was pleased to hear that some work has been done to make this interact with JavaFX. The solution presented though, was less than ideal. It had some duplication of configuration where the class name of the controller needed to specified in the FXML as you would expect but also when loading it. After a brief fumble through FXMLLoader's methods I found #setControllerFactory. By passing this wafer thin adapter below as a parameter to #setControllerFactory, any controllers will be instantiated via Guice.

JScrollPane Repainting Problems

I was having some problems with the JScrollPane not repainting my custom component properly when scrolling.  The results were that the area that was now visible because of the scroll was painted unevenly and looked blurred.  This seems to happen when a component is doing some custom rendering inside a JScrollPane and using the clip to find out what area it should paint.  To fix this call the following methods on your JScrollPane:

scrollpane.getViewport().setScrollMode(JViewport.SIMPLE_SCROLL_MODE);

In the modern swing implementation there is some 'clever' caching of the content view to improve performance which seems to be the problem.  Setting SIMPLE_SCROLL_MODE will force the scroll pane to repaint it's entire contents while scrolling.  Although this is less performant, it at least shows the correct results with minimum fuss.

Safely Removing an Element While Iterating a List

If you want to remove an element while you're iterating a Java ArrayList then you have some problems.  First off, if you use an iterator then the list will most likely throw a ConcurrentModificationException.  If you use a classic style for loop then when you remove an element the list will no longer be the same size or have it's other elements at the same position as when you started iterating which might cause an error, logical or otherwise.

The simplest way to do this is to iterate backwards over the list.  This means when you remove an element, the other elements will not get reordered.  The method below demonstrates:

    public void iterateBackwards()
    {
        List list = new ArrayList(Arrays.asList(1,2,3,4,5));
       
        for (int i = list.size() - 1; i >= 0; --i)
        {
            System.out.println(list.remove(i));
        }
    }

This prints....

  5

  4

  3

  2

  1

Fastest Way to Delete a Directory In Windows

I had a load of large directories (several GBs) which I needed to delete so that I could meet the minimum free space requirement for Windows to run the defragmentation tool.  Unfortunately, Windows Explorer is unusably slow for this as it first scans the whole directory so that it can show the estimate and then copies everything into the recycle bin or not if the directory is too big.

Below is some batch script to delete a directory very fast.  The code is on stack overflow here where the commenter Hugo bench marked it at 3 times faster than rmdir which itself is a hell of a lot faster than deletion via Windows Explorer.

bestrmdir.bat

rem to use, drop in your c:/windows/system32 directory for this to be usable anywhere

rem in then command line type 'bestrmdir MyDirectory' to delete MyDirectory

del /f/s/q %1 > nul 

rmdir /s/q %1

Typed Integers

I really like the idea of typed integers. For example, if you a number representing the number of cats and a number representing a number of dogs it is possible to add these together.  In a typed integer world it would not be possible and throw an error upon compilation.  I think this would be a big improvement for API safety and also a method of documentation.  There is nothing worse than a method with a structure like #foo(int, int) which is difficult to understand without reading or documentation every time.

In general use this would be suicide in a managed language where the overhead of an object for every int could mean a huge jump in memory usage and GC wouldn't be worth the improvements.  The actual implementation in Java would also be extremely ugly, for example:

class EventID {

  private final int id;

  public EventID(int anID) {

    id = anID;

  }

}

The class doesn't even contain basic methods like a way to get the ID, compare or hash it which with an int you get for free and it is still a fair wedge of code.  Lombok's @Data annotation can really improve classes like this and make the language feel more declarative.

I have found the best language to implement this is c++, the following is my proof of concept implementation:

template
class TypedInt
{
public:
  const int value;

  TypedInt(int v) : value(v) {};

  TypedInt operator + (TypedInt num)
  {
    TypedInt result(value + num.value);

    return result;
  }

  TypedInt  operator = (const int other)
  {
    TypedInt result(other);

    return result;
  }
};

This allows the following usage:

struct eventid {};

typedef TypedInt EventId;
  

EventId i = 2, j = 3;

EventId result = i + j;

Overall this feels natural to use.  If I was actually using class then the other operators such as == could be implemented without problems. 

One of the benefits of c++ is that an object wrapping an int does not take more memory than an int would, during compilation this abstraction is apparently stripped away.

Compatible Images

A couple of weeks ago I was tasked with improving the performance on a legacy java swing application that was really crawling.  It would repaint the whole screen on any event, which it could receive a lot of, and painting the whole screen was really slow.  By using compatible images instead of the default kind of image read by ImageIO, the painting code was sped up by a factor of 70 and the event frequency was a non-issue.  The whole change was about 10 lines of code, much less than that to make the software use the graphics clip.

If you are having an issue with graphics performance make sure you try compatible images before trying anything else since it is so easy to introduce.

Before deploying to site I was worried that the speed up I was seeing was in some way down to my dev machine working well with compatible images so I wrote the tool below to test if the improvement was the same which it was.  Some notes:

1. The tool requires an image.  When I attempted to create an in-memory image there was no difference in run times.
2. Results are printed on the command line so remember to use java.exe and not javaw.exe
3. I went to some effort so that this program was a single class to make it easy for the support guys to run.  I don't usually like to implement Runnable on a class that isn't just a Runnable.

How Much Architecture is Enough?

In the past I have often wondered if I have provided enough architecture for the software team to work to.  Conversely I always like to leave enough room to maneuver for the guys to do design for their own parts of the software, I would really hate it if I was working on a project where everything was defined and all I needed to do was "code in the blanks".

The idealist in me wants to provide just enough architecture to start work at all, and emergency design will do the rest.  I have found this is not how great software is made.  When everyone is left somewhat to their own devices, the code starts to lose coherence.  Everyone will code their own mini framework that will not interact well with other stories or allow extension for all requirements.  This is natural as everyone might not be aware of all requirements for the project.

The answer to this is pretty simple in practice, whenever two or more stories meet the following criteria you need some architecture:

1. Other code interacts with the product of a story.  
For example some kind of remote or local service or data.  In my current work we do a lot of inter-process communication.  This is really a technical detail, web services or in-process framework meets the same criteria.  Whatever it is, it needs some documented architecture.  The documentation part here is important, it should be simple to find look-up the exposed interfaces with arguments including valid values, errors etc.  This rule tends to reduce errors and confusion (more errors) occurring from not knowing about or not understanding .

2. Two or more stories are similar

For example, two kinds of message or input that could be handled in the same way or are similar enough to be categorised together.  Polymorphism and/or inversion of control can be used to have a central mechanism handle these in a reusable way.  This rule tends to reduce the overall amount of code and improve code quality. 

These two scenarios are sum up pretty much of every time where I had wished I had provided some architecture before work commenced

Some of My Coding Habits

This is a brief description of my coding habits including indentation, new lines and other things that I don't usually see although I think it makes code clearer.

Variable declaration and assignment on separate lines
Example:

JLabel nameLabel;

nameLabel = new JLabel("Name:");

nameLabel.setBackground(Color.RED);

Although I didn't take up SmallTalk seriously I did come away with, among other things a new coding habit.  SmallTalk variables are always declared in pipes i.e. | myVar | before use.  In Java I use this, especially when the object needs one or more method calls after it is created.  It is almost like a paragraph header and allows the code on the left hand side to match up which I find quite easy to read.

Static imports

No need for an example here, I just really like static imports... in moderation.  They are particularly effective for enums where the enum name might be repeated over and over again.  It is also quite safe to use enums in this way because type safety will not allow you to do anything silly such as EnumA.RED == EnumB.RED without giving a compilation error.

Static imports are quite bad for importing primitive constants, it is much easier to make a mistake without the context of the class name from where it came.  For example, two different integers might mean very different things.  A number of cats and a number of dogs which should not be added together. 

It is quite rare for me to write static methods so this isn't usually an issue.

Returning new
Example:

public String findDayName(int dayIndex)

{

  assert dayIndex >= 1 && dayIndex <= 7;

  switch(dayIndex)

  {

    case 1: 

      return "monday";

    case 2: 

      return "tuesday";

    default: 

      return null;

  }

  // and when using an if statement

  if(dayIndex == 1)

    return "monday";

  else if(dayIndex == 2)

    return "tuesday";

  else

    return null;

}

When using a reasonable size switch or if statement I typically bring this out into a separate method with no variables other than the data needed to perform the switch.  Even this is not necessary if the data is a class field. There is no variable that gets assigned to so no need to worry about a name, or maintain the name and the method name make sense together.  If statements also do not have brackets to cut down on needless lines and white space which in this case would just make the code more bulky and difficult to read.

The if statement is reminiscent of Erlang guarded methods.

Resolving Bugs

There are at least three roles in a bug report.  The first two are the bug reporter or owner of the bug who is equivalent to the Customer or Product Owner and the Software Engineer responsible for the bug.  An Architect, Technical Lead or Project Manager may exist who has more direct access to the customer, this is called QA in Bugzilla.  In certain cases the Engineer may need to escalate issues if they cannot resolve it themselves.  This can happen all the time, the Customer might raise a bug stating that functionality is missing when it is not in the requirements.  If this is not dealt with in a way that includes all stake holders, it could cause issues immediately or later on in the project at demo time.

Below is a table of resolutions and the circumstances in which they can be applied and by who.  The only possible option for the software engineer is to fix the bug or agree another resolution with the customer, or team lead.  This means that the bug will always be fixed or dealt with in a final way where the specification will be updated to reflect the new behaviour (or new understanding of behaviour) or at least a resolution agreed which should be traceable in the bug comments.  If the bug is not fixed and not agreed then it is bound to come back in the future starting with the line “I reported this months ago and it is still NOT FIXED”!

Fixed	Only the Software Engineer can set.
Invalid	Only the Customer can set.
Won’t Fix	Architect, Technical Lead or Project Manager can set based on agreement with customer.  Should be extremely rare.
Later	Never set
Remind	Never set
Duplicate	Software Engineer can set in the interest of efficiency.  Be careful that this is truly a duplicate and the customer is in fact not describing some other behaviour.
Works For Me	Never.  Negotiate with the Customer for them to set the bug to Invalid instead.