The release is too big!

It isn’t unusual for me to release at least one new update a week for the Costs to Expect API and website. I find it significantly easier to manage and test with small iterative releases. As the only developer on this project, anything that makes my life simpler is a plus.

Occasionally a release has to touch a substantial part of the Costs to Expect API, v2.02.0 is an example of that. I need to review every section of code relating to items.

I intended to split the development work for multiple item types across two versions, v2.00.0 and v2.02.0. The database gets upgraded, and then I hit the code and expose the feature.

In planning, I appear to have overlooked summary routes. Rather than extend the design I developed to incorporate item summary routes; it makes sense to modify it slightly, given the ‘new’ requirements. If I soldiered on and upgraded the item summary routes, I suspect I would want to refactor them in a few weeks. Rather than face that situation, I feel it makes sense to go back to the drawing board and ensure I develop a design that will work going forward.

Post-release of v2.02.0, I will rework the current design, refactor as necessary and then add support for multiple item types to the item summary routes.

I gave myself one to two weeks to develop support for multiple item types. It turns out, two weeks is probably about right given the increased (actual) scope.

Costs to Expect API, v1.24 -> v2.00

Before I start developing the Costs to Expect app, I need to enhance the Costs to Expect API. Two significant features are going to be in the next version of the API, permitted users and multiple items types.

Permitted users

The Costs to Expect API supports authenticated users; however, the assumption is all authenticated users have the same data access. This system works at the moment because there are only two users; this works at the moment as there are only two users. The app needs to support multiple users with their own private data, so I’m going through the app and adding the necessary user links.

Multiple item types

The `item` table is set up for child expenses; that was all I needed for the initial development of the API. Looking forward, the API needs to support more complex item types. In the next release, I’m going to add support for multiple items types with a long term goal being support for custom item types.

The item type will need to be selected when you create a resource type. If you are tracking expenses you will choose `expense item type`, for budgeting you would choose `budget expense type`, and for a list, you would select `list item type`.

Custom item types will be coming; at this time, I can’t confirm when. I need to develop the budgeting and forecasting system; after that, I will be able to give a rough timeline.

Version 2.00

The features above are significant, far more significant than any additions to the API since its release in 2018. I feel that these additions are enough to justify a v2.00 release. 

A bump in the major version means I’m not going to support upgrades from v1.24 to v2.00. Upgrading from v1 to v2, will be possible. To upgrade you will need to export and import your existing data into a new instance of the API. I suspect there will be a join/relationship table between `items` and their data. Until v2.00 of the API is out, I am unable to confirm the exact setup; it is still in development.

Custom item types are unlikely to be supported in the v2.00 branch, I suspect, they will be the notable feature for v3.00. Unlike the jump from v1 to v2, I can guarantee there will be an upgrade path from v2 to v3, the existence of the app and website (Costs to Expect service) will require it.

Open Source Costs to Expect website

I have decided to Open Source the Costs to Expect website.

I planned the website and the app for the service as a single app. Over time, I’ve concluded that although it may create additional work, it makes sense to separate the service app (app.costs-to-expect.com) from the public website (www.costs-to-expect.com).

Due to the limited requirements, the public website will never be an excellent showcase of the Costs to Expect service.

Until we decide I need help, I’m creating the Costs to Expect service by myself. Managing three apps is more involved than two; however, there are pluses to separating the products.

The public website is a minor part of the service; I have numerous features planned; however, they are much lower in priority than developing the app. I don’t want to be in a position whereby the website is restricting the development of the app.

For the next three to four months, I am moderately confident that I can develop the app and update the API without negatively affecting the website. I have request tests in place (thank you Postman) that will immediately notify me if I make any change to the API, which affects the website.

API endpoints, filtering, sorting, searching, limiting and summarising

The Costs to Expect API is developing slowly; features get added as I need them, v1.15.0 is an exciting release, it will be the first time an endpoint in the API can be filtered, sorted, limited, searched and summarised.

I’ve written previously about summarising collections, with the release of search in v1.15.0 it makes send to describe all options supported by the items (/v1/resource-types/[resource-type]/resources/[resource]/items) collection on the Costs to Expect API.

By default, the items collection returns all child expenses in reverse chronological order, ‘all’ and ‘reverse chronological’ are sensible defaults; however, the API is limited if the collection always returns everything, you need to be able to filter, sort, limit, search and summarise.

The Costs to Expect API supports filtering, sorting, limiting and searching via parameters appended to the URI, collections are summarised by prepending /summary/ to the URI. Summary endpoints support the same filtering and searching options as their companion collections, limiting and sorting don’t make any sense in the context of a summary.

A URI can quickly become unreadable when multiple GET parameters are assigned, to attempt and improve readability I have grouped sorting and searching.

The OPTIONS requests for the Costs to Expect API detail all the supported GET parameters, in addition to giving an overview of each parameter, the OPTIONS requests list which fields and sortable and searchable. The sort and search parameters both allow multiple conditions.

Limiting a collection is handled as expected, there is an offset parameter to define the start record and a limit parameter to define the number of records requested.

Filtering a collection is handled via individual parameters, filtering is more common than searching and in my opinion needs to be the most verbose, when reviewing a URI the filtering parameters should be more visible.

This solution won’t work for everyone; however, if you need to support filtering, searching, sorting, limiting and summarising, I think it provides an excellent level or readability while still being practical.

Self-documenting APIs

During my professional career I’ve worked on many APIs, however, I’ve never been responsible for the design, development and support of an API of more than trivial scale, when designing my own I wanted to do ‘as good of a job’ as I possibly can.

I’ve heard the term self-documenting APIs and never really looked into it, regardless, I’m going to describe what it means to me, hey, it is the internet, we all have opinions.

To me, there are five points.

  • The initial entry route or an obviously named route should display all the endpoints.
  • The initial entry route or an obviously named route should show the current version of the API as well as provide links to the README, CHANGELOG and anything else useful.
  • There should be a changelog route to describe all changes and updates to the API.
  • An OPTIONS request should exist for every route. The OPTIONS request should detail the purpose of the route, all possible verbs, as well as any fields and parameters.
  • No redundant information in either the payload or the response, for example, the payload should not be wrapped in an envelope, the verb is the envelope.

As a bonus, API versioning should be controlled via the route, any payloads or responses should be free of any version information.

Releasing…works

I have a long-term project, Dlayer, that I very much believe in, but after many false starts, it still doesn’t exist, it will though, I’m due to restart it later this year with a slightly different focus 🙂

Why did I never finish or release the project? I never managed to hit what I considered to be the minimum viable product (MVP), my idea of the MVP for Dlayer was too vast, and far too much for one developer, I never managed to get enough done.

Early last year I needed to develop a small library, my PHP Quill renderer, I developed just what I needed at the time and quickly released it, within six months I bumped it to v1.00. I knocked it to v1.00 because I was confident in the library, it had warts, but people seemed to be using it, it wasn’t just solving my personal problem.

I’m not suggesting tagging it with a v1.00 release number made it slightly popular, I’m saying that because I was confident enough to add the v1.00 release number it became more popular.

Tagging the library with v1.00 meant I had to handle it differently, I needed to ensure the README and CHANGELOG were accurate, and up to date, tests are required, and I needed to learn how to handle the pull requests I started receiving. All of this made the library better, as I’ve said in the past, nothing makes your code better than knowing there is a minuscule chance other developers will look at your library.

During the summer I released two apps, both v1.00, the REST API for Costs to Expect and the companion Web app. Neither of these apps is complete. The REST API is missing quite a few features; however, it supports enough to get the job done, replace my spreadsheet and allow my wife to submit data.

I’m not saying anything new; there is a blog post by Jeff Atwood referencing a blog post by Chuck Jazdzewski stating our job is to ship. Knowing that is one thing, doing it is another, I wish I had had the confidence to ship earlier, and I hope I keep it up.

Open Source is awesome, PHP Quill Renderer v3.01.0

A few hours after arriving at work I got an email from GitHub letting me know about a posted issue on one of my Open Source libraries, turns out I had missed a couple of things in my testing.

When I create a library, I have a use case in mind; typically I’m trying to solve a problem specific to one of my projects, having an extra set of eyes reviewing my libraries and using them differently is terrific.

I’m always happy to receive bug reports; each fix improves the library. Hopefully, there will not be too many more but if there are I will jump on them to fix the issue.

I have released v3.01.0 of my PHP Quill Renderer; Parser::load() wasn’t resetting the deltas array, and the Compound delta was incorrectly trying to handle images with multiple attributes assigned.

PHP Quill Renderer v3.00.0

I have released v3.00.0 of my PHP Quill renderer; the rewrite only took three weeks, I expected it to take a little longer, but once I put fingers to keyboard, it came together very quickly.

I had a significant head start when I started the rewrite, I had the design down on paper, and I had the unit tests from v2.03.1, it can never be over-stated, when your library has good test coverage working on an almost complete rewrite is a joy.

I’m hoping v3.00.0 will be well received; I’ve had more emails than I would have expected from other developers asking for progress, raising bugs and thanking me, developers appear to be using it.

The new design is much more flexible than v2.03.1; it should make it simple for me to support the rest of the features in Quill without continually adding hacks, a colleague at work helped pushed me in the right direction.

I now need to work on the v3.01.0 release to tidy up a few issues and then create a simple demo site to show live use; the one-page website will display a Quill editor, the delta after submission and the output in the requested format as well as the PHP code itself.

v3.00.0 supports all the features present in v2.03.1; it fixes the issue with paragraphs sitting next to lists and paragraphs being ignored and includes additional test coverage.

Row and column view helpers

Over the weekend I released updates to three of my libraries, Zend Framework 3 view helpers has been bumped to v1.02.1, Zend Framework 3 view helpers code completion is now v1.07.0, and Bootstrap 4 helpers is now at v0.04.

I added row and column view helpers to the Zf3 view helpers library, the other libraries have been updated to include the new additions.

For an overview of all my libraries, visit Transmute-Coffee.com, to get the code, go to GitHub.

Library updates

I’ve just pushed three library updates, the Bootstrap 4 helpers library is now at v0.03, the Zend Framework 3 view helpers library is at v1.01 and the sister library which provides code completion is now at v1.06.

I’m slowly adding support for additional Bootstrap components, this time the alert component, details and links can be found at Transmute Coffee.