Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(images): add support for Core Lightning v23.08.2, v24.02.2 & v24.05 #879

Merged
merged 9 commits into from
Jun 18, 2024

Conversation

jamaljsr
Copy link
Owner

Closes #772
Replaces #774

Description

Adds support for Core Lightning v23.08.2 & v24.02.2.

I've migrated to using the Core Lightning embedded REST API plugin clnrest from previously using the third-party c-lightning-REST plugin. Since older versions of CLN do not support the clnrest-port flag, I have dropped support in Polar for versions prior to these two.

Screenshots

image

Copy link

codecov bot commented Apr 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (2a8b97e) to head (f5b8580).

Additional details and impacted files
@@            Coverage Diff            @@
##            master      #879   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          141       141           
  Lines         4609      4663   +54     
  Branches       897       902    +5     
=========================================
+ Hits          4609      4663   +54     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jamaljsr
Copy link
Owner Author

I'm going to hold off on merging this until right before shipping the next release. The removal of older CLN nodes from nodes.json would break existing Polar users that could apply the out-of-band node version updates.

@toneloc
Copy link

toneloc commented Apr 26, 2024

Awesome! I would like to help test building the CLN Docker images.

Please let me know how I can best help test.

@jamaljsr
Copy link
Owner Author

Hey @toneloc, the images have already been built and pushed to Docker Hub. The instructions for how I build the images are in the docker readme.

If you want to test them out in Polar, you'll need to be running the the code in this PR branch because the new images won't work in v2.2.0. You'll need to setup your dev environment to run Polar from source code. The instructions are in the CONTRIBUTING doc. Just be sure to checkout this branch.

@toneloc
Copy link

toneloc commented Apr 26, 2024

Awesome thanks. FYI, I had to remove --openssl-legacy-provider in package.json to get it to work on my Mac M2 ... but it worked.

After that I was able to start up a Polar network with two v24.02.2 CLN nodes and create a channel and send payments back and forth. All via UI.

If you want me to test anything in particular please lmk. Will let you know if I see anything else notable.

Nice work. Next I want to try to create custom Docker image so I can add in a CLN plugin. Any tips on that please lmk.

@jamaljsr
Copy link
Owner Author

@toneloc Thanks so much for testing and confirming it works for you. That is very helpful.

FYI, I had to remove --openssl-legacy-provider in package.json to get it to work on my Mac M2 ... but it worked.

What version of NodeJS are you running? I've seen issues related to this flag when you aren't running v20.

Next I want to try to create custom Docker image so I can add in a CLN plugin. Any tips on that please lmk.

Getting your plugin into the docker image should be pretty easy. This is how Polar currently installs the c-lightning-REST plugin into the /opt/c-lightning-rest/ dir in the image.

# install nodejs
RUN apt-get update -y \
&& apt-get install -y curl gosu git \
&& curl -sL https://deb.nodesource.com/setup_14.x | bash - \
&& apt-get install -y nodejs \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# install c-lightning-REST API plugin
RUN git clone https://github.com/Ride-The-Lightning/c-lightning-REST.git /opt/c-lightning-rest/ \
&& cd /opt/c-lightning-rest \
&& npm install \
&& chmod -R a+rw /opt/c-lightning-rest \
&& mv /opt/c-lightning-rest/clrest.js /opt/c-lightning-rest/plugin.js

You'd just need to get your plugin's files into the image via git clone or COPY. Once the files are inside of the image, you can update the node's startup command in Polar to include the path to your plugin. Similar to this:

--plugin=/opt/c-lightning-rest/plugin.js

@toneloc
Copy link

toneloc commented Apr 27, 2024

What version of NodeJS are you running? I've seen issues related to this flag when you aren't running v20.

I am running Node v20.0.0.

Awesome thanks and cheers. I was able to build the image and test. This is further then I've ever been so kudos to you.

In the Dockerfile I had to remove the line COPY .bashrc /home/clightning/.bashrc and manually change v${CLN_VERSION} to v24.02.2 (and of course also add git clone and CP and MV install lines to Dockerfile, as you mention).

Note that when creating channels the green lines didn't show up between custom nodes, not sure why.

Did not have error without custom nodes. Nothing weird in CLN logs.

Anyway, error and screenshot pasted below. Cheers

[electron] 22:39:20.379 › Failed to sync the network TypeError: Cannot convert undefined or null to object
[electron]     at Function.keys (<anonymous>)
[electron]     at http://localhost:3000/static/js/main.chunk.js:39919:12
[electron]     at Array.forEach (<anonymous>)
[electron]     at updateChartFromNodes (http://localhost:3000/static/js/main.chunk.js:39918:24)
[electron]     at Object.fn (http://localhost:3000/static/js/main.chunk.js:35937:91)
[electron]     at processTicksAndRejections (internal/process/task_queues.js:93:5)
[electron]     at async http://localhost:3000/static/js/main.chunk.js:5693:7
Screenshot 2024-04-26 at 10 40 26 PM

@jamaljsr
Copy link
Owner Author

Oh interesting. From looking at the code involved in that stacktrace, I suspect that this error is not the root cause, but a result of the chart links object being undefined, which shouldn't be possible. Is this the first error displayed in the logs immediately after opening a channel?

Would you mind sharing logs?

  • The files in ~/.polar/logs. Please stop Polar, delete the existing files, then start Polar and reproduce the issue so the logs are isolated.
  • The DevTools console logs would be helpful as well. If you run yarn dev, the DevTools should automatically open. If you go to the Console tab then right click anywhere on the logs and choose "Save as..." to save the logs to a file that you can share.

@toneloc
Copy link

toneloc commented Apr 27, 2024

Look like on restart the line showed up!

Then I created a channel and it is working now. Not sure what happened.

FYI:

Polar logs:

polarlogs.txt

And front end:

localhost-1714237419178.log

The Polar logs look okay. The front end logs may show you something! I'll let you known as I play around.

@michaelWuensch
Copy link

I can also confirm that it works with the new CLN versions. Didn't encounter any issues so far. 👍

@jamaljsr
Copy link
Owner Author

jamaljsr commented May 5, 2024

@michaelWuensch Thanks for the feedback. 👍

@jamaljsr jamaljsr force-pushed the cln-24.02.2 branch 2 times, most recently from 4875f6a to 44a3bc0 Compare May 16, 2024 03:22
@darioAnongba
Copy link

Hi @jamaljsr and thanks for this awesome features. Any idea when this will be merged ? Otherwise, how to use this version of Polar even before the release? Currently struggling a lot to run CLN 24.02 with clnrest locally, using Polar would help a lot.

@jamaljsr
Copy link
Owner Author

jamaljsr commented Jun 5, 2024

Hi @darioAnongba, I'm planning on packaging up a release sometime next week. I'm trying to wait for a few of the currently open PRs to get merged.

If you'd like to locally package a build of Polar with the changes from this PR, I mentioned how to do this in #879 (comment).

@darioAnongba
Copy link

darioAnongba commented Jun 6, 2024

thanks! I'll try to run it on my own with your comment.

Edit: just letting you know that this PR no longer compiles because the version of the package tapd-api that is used here has been removed in npm. I tried running the version in npm 0.3.99-alpha.pre1 but it has breaking changes so the code doesn't build anymore.

I'll better wait patiently.

@michaelWuensch
Copy link

Can you also add CLN 24.05 with the upcoming release?

@jamaljsr jamaljsr changed the title feat(images): add support for Core Lightning v23.08.2 & v24.02.2 feat(images): add support for Core Lightning v23.08.2, v24.02.2 & v24.05 Jun 8, 2024
@jamaljsr
Copy link
Owner Author

jamaljsr commented Jun 8, 2024

@michaelWuensch I added support for v24.05 here as well.

@jamaljsr jamaljsr force-pushed the cln-24.02.2 branch 2 times, most recently from dc88a5f to e0b692f Compare June 8, 2024 08:47
@niteshbalusu11
Copy link
Contributor

Can this be merged? :)

@niteshbalusu11
Copy link
Contributor

niteshbalusu11 commented Jun 12, 2024

Actually I built this branch myself and i am getting these errors for cln and lnd and can't get the containers to start.

lightningd: --log-file=/home/clightning/.lightning/debug.log: Failed to open: No such file or directory
failed to load config: ValidateConfig: log rotation setup failed: failed to create file rotator: open /home/lnd/.lnd/logs/bitcoin/regtest/lnd.log: no such file or directory

@jamaljsr
Copy link
Owner Author

@niteshbalusu11 What OS and docker version are you running. I tested on Mac, Windows, and Linux and didn't have any issues.

@niteshbalusu11
Copy link
Contributor

niteshbalusu11 commented Jun 12, 2024

@niteshbalusu11 What OS and docker version are you running. I tested on Mac, Windows, and Linux and didn't have any issues.

It's only happening with latest versions of LND and CLN btw.

MacOS Arm chips

docker compose version
Docker Compose version v2.27.0-desktop.2
docker version
Client:
 Cloud integration: v1.0.35+desktop.13
 Version:           26.1.1
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        4cf5afa
 Built:             Tue Apr 30 11:44:56 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.30.0 (149282)
 Engine:
  Version:          26.1.1
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       ac2de55
  Built:            Tue Apr 30 11:48:04 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

@jamaljsr
Copy link
Owner Author

Ok, I'm running on an ARM Mac. Those are filesystem errors you're getting. My hunch is that Docker is misbehaving. Try restarting Docker and/or your computer. That will likely resolve the issue.

@niteshbalusu11
Copy link
Contributor

Ok looks like it was docker misbehaving. Seems to be working fine now. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants