Module-builder and custom module setup

Community Support
#1

Hi.

We’re working on a custom module with skills, actions, hooks etc.
We do this in a separate project/repo “botpress-module”, which is located next to a local botpress installation. We put a symlink into botpress/modules under a specific name to make our custom module available in botpress.

  • botpress-module <- our “foo” module, watched and recompiled via module-builder
  • botpress
    • modules
      • foo <- symlink to …/…/botpress-module

To build our module, we depend on "module-builder": "file:../botpress/build/module-builder"

This does not work with the latest versions because of this commit: https://github.com/botpress/botpress/commit/35f23a1980f186c7b8bd92b3076e739c63afb8f9

The commit introduces some hardcoded overrides on loaded tsconfig settings, assuming only the default use case and effectively ignoring our own use-case and configuration. During compilation, the typescript compiler fails due to incorrect relative paths.

One solution I found was to somehow provide our own module-builder and use that - be it a fork, or as in our case, a local package in a yarn workspace - where we commented out the overrides introduced by said commit.

  • botpress
  • packages
    • module-builder
    • botpress-module

However, for now we simply stick with botpress 12.5.0 anyways, which is before the commit and change. We might fall back to the modified module-builder, but it’s really a suboptimal situation. Also, we don’t have more capacity to investigate and debug that particular issue, so I hope for some help in the forum here.

What is the recommended way of developing your custom module, where would you place the module in the filesystem? How would you make it available to botpress?
The approach we currently use - keep module outside of botpress, link it into botpress/(internal-?)modules - is it a silly approach, or basically the right direction?

Thanks in advance, I appreciate any tips and feedback!

#2

Hi, @loopmode- i wonder if You were able to find some solution for this? Into what did the build process of Your custom module evolve?

#3

Hi @arcijsg

We ended up using a yarn workspaces monorepo with various packages, one of them being the botpress module itself, and another one being an actual copy of the botpress module-builder where we commented out some lines that would otherwise override tsconfig settings of the module.

Botpress is in the same repo, but it is not a workspace package as to avoid dependency problems.

The botpress module is still symlinked into botpress/internal-modules, this works okay.

Since this is a work/job project, I can’t just share the code. But I might be able to set up something similar on GitHub…if I find the time, that is. It’s a workable setup all in all.

Also, feel free to ask more specific questions here, we can continue the thread (I had forgotten about it already :))

#4

Thank You, @loopmode, for sharing Your solution!

It would be interesting to see a proof-of-concept of that approach somewhere in Github, if You happen to have a little bit of free time and interest for that?

In our team, we currently ended up having a multi-stage Docker workflow - where at first we clone Botpress source, then take module-builder and botpress sdk source and build them separately.

Having done that, we mount source code of our custom botpress module(s), supply module-builder binary for them and are able to build and package with yarn a .tgz file, which can then be either released to production server, or - in development environment, we use official Botpress binary image, and copy over the .tgz file we built to /botpress/modules folder, and restart botpress server.

It works quite fine as soon as the docker is built for the first time (on subsequent changes we can just stop the docker and build it again, which takes under two minutes - yarn build and yarn package are the only steps which are not cached by docker engine), but it still feels a bit clumsy and…wrong :slight_smile:

So, i am looking for a nicer and better ways to speed up development cycle of our custom botpress modules, in a way which would allow our developers to not depend on node 10.11 (as required by Botpress 12.10.x) being available on their workstations (we have other projects, which require various different node versions). Also, it kind of helps to mimic production environment as close as i can currently imagine, but - i’m still hoping to find a more fluent way to rebuild and see changes, without losing benefits this setup currently offers us.

It is really interesting to know how other people are dealing with their development setups for botpress custom modules - thanks for sharing Yours!