Skip to content

Conversation

@zachfedor
Copy link
Collaborator

@zachfedor zachfedor commented Jun 6, 2025

This PR adds the ability to create a new Filecoin wallet via CLI or API endpoint using offline keypair generation, saving the new wallet address and private key to the database as if it was imported.

It also adds a mechanism for versioned migrations as well as commands to facilitate this:

# This will implement any missing migrations after user updates to new version
$ singularity admin init

# These will allow a user/dev to traverse through migrations for troubleshooting
$ singularity admin migrate up
$ singularity admin migrate down
$ singularity admin migrate to <migration_id>

# And this will print the current migration ID
$ singularity admin migrate which

Partially completes #469

masih and others added 30 commits July 7, 2023 22:20
Instead of `assert`, use `require` in order to fail fast when an error
occurs during testing.

Fix error checks to use `NoError(err)` instead of `Nil(err)`.

Avoid panicking during tests, by replacing the `cid.Must` calls with
fatal test error.
Refactor `Must*` functions to never panic during normal execution, and
instead return error.

Replace `cid.MustParse` with `cid.Decode` to avoid panicking.

Remove redundant wrappings of `cid.MustParse`.
Consistently optimise imports across all files and consistently format
files.

In go code, group SDK package imports together for better readability of
non-sdk imports.
* Change swaggo annotation to unexported functions
* Regenerate CLI reference
* Regenerate swagger.json

This solves #54
* Use a more reliable cron library to manage deal making push mode
* Work in progress. Check in unfinished code before a long week leave.
Co-authored-by: Xinan Xu <39632914+xinaxu@users.noreply.github.com>
Co-authored-by: web3-bot <web3-bot@users.noreply.github.com>
So that the front end engineer from FF can be unblocked
Also, unblock the CI actions
Reduce test time so CI runs faster
Basic coverage for deal making service
This creates a tester tool that can be used to export prepared CAR files
to a folder structure so that the tester can verify the correctness of
the preparation process.
* folder CID -> folder
* file CID -> file
* Black box tests for now to unblock testers.
# Goals

Move more code into handlers, so we can setup a library interface --
this removes the last major piece of logic code in the API

# Implementation

- HandlePostSource to CreateSourceHandler
- A few cleanups around handler errors
- Move PushItem under datasource directory (this is where one would
expect it to be given the API syntax)
- Fix tests to work with the handler
- Adds a check on the parameter names to make sure they match the
datasource type (reject inappropriately named params)
- 

# For discussion

This also fixes the build errors I introduced when I merged #85
Add CI workflows to build the Singularity container on each PR, and publish it onto ghcr.io for convenience on merge to main and tagged versions.
Update code to use `any` keyword instead of `interface{}` since the go mod version is 1.19.

The changes here are purely cosmetic and have no effect on net functionality.
Allow only once instance of deal maker and deal tracker. Later we will
also only allow one instance of metrics reporter.
- [x] Testnet/Calibnet can be enabled by setting
  * LOTUS_API
  * MARKET_DEAL_URL
* The genesis timestamp will be figured out by calling
lotus.ChainGetGenesis, or default to mainnet genesis timestamp if
LOTUS_API is the default GLIF mainnet API endpoint
- [x] Remove places that set address.CurrentNetwork
`go generate` is run during CI to check if any file is not updated.
Below files are automatically generated
* data source add swagger annotation will be generated from supported
Rclone backend options
* swagger file will be generated from swagger annotation
* CLI reference will be generated from capturing the help page for all
subcommands
* WebAPI reference will be generated from the swagger file

Plus some other fixes
* Changed the documentation to point to the ghcr.io for installing via
docker
* The doc generated will use no longer use current user as the default
value but a string literal '$USER' instead
* The default sqlite database is the one from the current directory
instead of assuming there is a home directory
* Fix a race condition from zstd Read and Close at the same time by
wrapping it in a thread safe struct
parkan and others added 15 commits February 15, 2025 01:12
fixes for linter env and lint errors

---------

Co-authored-by: Arkadiy Kukarkin <arkadiy@archive.org>
- Add init function to create full current schema on brand new clean
databases and run missing migrations on any databases using old
auto-migrate strategy with existing data.
- Add CLI commands to migrate up, down, or to a specified version by ID
- Add utility functions to get list of migration IDs ran on current
database, check if migration has run by ID, etc.
Fix struct tag ordering in handler/storage/types_gen.go to match
the current code generator output. This resolves CI failures where
go generate was detecting differences between committed and generated files.
Copy link
Contributor

@Sankara-Jefferson Sankara-Jefferson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zachfedor I am experiencing this error, everytime I run the commands below. I had to switch from sqlite to postgres before I pulled the new PR.


jeffersonsankara@Jeffersons-MacBook-Pro singularity % ./singularity admin migrate which
2025-06-11T16:01:15.928-0700    INFO     database        database/connstring_cgo.go:35   Opening postgres database
2025-06-11T16:01:15.958-0700    WARN     model   model/migrate.go:187    Skipping SCHEMA_INIT check due to SKIP_SCHEMA_INIT env var
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x1057c8310]

goroutine 1 [running]:
github.com/data-preservation-programs/singularity/model.(*migrator).GetMigrationsRun(0x0)
        /Users/jeffersonsankara/singularity/model/migrate.go:153 +0x30
github.com/data-preservation-programs/singularity/model.(*migrator).GetLastMigration(0xc001040f60?)
        /Users/jeffersonsankara/singularity/model/migrate.go:162 +0x13
github.com/data-preservation-programs/singularity/cmd/admin.init.func5(0xc000b04580?)
        /Users/jeffersonsankara/singularity/cmd/admin/migrate.go:85 +0x10a
github.com/urfave/cli/v2.(*Command).Run(0x10a62c200, 0xc000b04580, {0xc00104cb70, 0x1, 0x1})
        /Users/jeffersonsankara/go/pkg/mod/github.com/urfave/cli/v2@v2.27.3/command.go:276 +0x7e2
github.com/urfave/cli/v2.(*Command).Run(0x10a62bc80, 0xc000b04440, {0xc000787ec0, 0x2, 0x2})
        /Users/jeffersonsankara/go/pkg/mod/github.com/urfave/cli/v2@v2.27.3/command.go:269 +0xa65
github.com/urfave/cli/v2.(*Command).Run(0x10a62aec0, 0xc000b04340, {0xc001040d80, 0x3, 0x3})
        /Users/jeffersonsankara/go/pkg/mod/github.com/urfave/cli/v2@v2.27.3/command.go:269 +0xa65
github.com/urfave/cli/v2.(*Command).Run(0xc001050160, 0xc000b04100, {0xc0001fe200, 0x4, 0x4})
        /Users/jeffersonsankara/go/pkg/mod/github.com/urfave/cli/v2@v2.27.3/command.go:269 +0xa65
github.com/urfave/cli/v2.(*App).RunContext(0x10a632680, {0x108ca2d30, 0x10a678c20}, {0xc0001fe200, 0x4, 0x4})
        /Users/jeffersonsankara/go/pkg/mod/github.com/urfave/cli/v2@v2.27.3/app.go:333 +0x5a5
main.main()
        /Users/jeffersonsankara/singularity/singularity.go:54 +0x225
jeffersonsankara@Jeffersons-MacBook-Pro singularity % 

That meant that I could not test wallet create command.

jeffersonsankara@Jeffersons-MacBook-Pro singularity % ./singularity wallet create 2025-06-11T16:04:10.278-0700 INFO database database/connstring_cgo.go:35 Opening postgres database 2025-06-11T16:04:10.334-0700 ERROR database database/util.go:80 INSERT INTO "wallets" ("actor_id","actor_name","address","balance","balance_plus","balance_updated_at","contact_info","location","private_key","wallet_type") VALUES ('','','f1qbmpjw7ii4pbpo6zidfzthou74ksufl274eznrq',0,0,NULL,'','','7b22507269766174654b6579223a224e6e714b455569394a54443947414a5472496d553764636c785350356b54545671705065783754455a6e733d222c2254797065223a22736563703235366b31227d','UserWallet') RETURNING "id" {"rowsAffected": 0, "elapsed": 0.013970125, "err": "ERROR: column \"actor_id\" of relation \"wallets\" does not exist (SQLSTATE 42703)"} ERROR: column "actor_id" of relation "wallets" does not exist (SQLSTATE 42703) jeffersonsankara@Jeffersons-MacBook-Pro singularity %

@zachfedor
Copy link
Collaborator Author

zachfedor commented Jun 12, 2025

@Sankara-Jefferson did you run ./singularity admin init first? I believe that's how the automigrations were run on previous releases, so that's where I implemented the logic to transition to versioned migrations if the database hasn't already.

But that brings up a good idea to have better error handling on the singularity admin migration sub-commands if the database hasn't been transitioned to versioned migrations yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.