amino's "unrecognized concrete types"

picture this: you're making a small utility to read transactions directly from the result of a /block query on an rpc node (see this example). you start writing something like this:

package main

import (
	"encoding/json"
	"fmt"
	"io"
	"os"

	"github.com/gnolang/gno/tm2/pkg/amino"
	"github.com/gnolang/gno/tm2/pkg/std"
)

func main() {
	data, err := io.ReadAll(os.Stdin)
	if err != nil {
		panic(err)
	}
	var blockData struct {
		Result struct {
			Block struct {
				Data struct {
					Txs [][]byte `json:"txs"`
				} `json:"data"`
			} `json:"block"`
		} `json:"result"`
	}
	if err := json.Unmarshal(data, &blockData); err != nil {
		panic(err)
	}
	for _, txBytes := range blockData.Result.Block.Data.Txs {
		var tx std.Tx
		amino.MustUnmarshal(txBytes, &tx)
		fmt.Printf("%+v\n", tx)
	}
}

something you may not know: when you have a []byte and you pass it to json.Marshal, it is encoded as base64. conversely, when you decode a string into a []byte, it is understood as base64.

then, when you run it, you get hit in the face by this error:

panic: unmarshal to std.Tx failed after 20 bytes (error reading slice contents: amino: unrecognized concrete type full name vm.m_addpkg): [...]

the first, rational answer to this message is despair. how is it possible that amino doesn't understand this? did the core team change the names of the messages? are they all incompatible now? are we alone in the universe?

the fact is, when you use amino, you need to use it in a similar way to encoding/gob. that is to say, when amino encodes something wrapped in an interface, it needs a way to understand what is the underlying type that it's encoding. we can see this, for instance, in the amino.MarshalJSON representation of the same std.Tx:

{
  "msg": [
    {
      "@type": "/vm.m_addpkg",
      "creator": "g1jg8mtutu9khhfwc4nxmuhcpftf0pajdhfvsqf5",
      "package": {
        "name": "hello",
        "path": "gno.land/r/pkg1",
        "files": [
          {
            "name": "a.gno",
            "body": "package hello\n\nimport (\n\t\"strconv\"\n\n\t\"gno.land/p/demo/avl\"\n)\n\nvar hello avl.Tree\n\nvar helloMap = make(map[string]string)\n\nfunc init() {\n\tfor i := 0; i < 1000; i++ {\n\t\ts := strconv.Itoa(i)\n\t\thello.Set(s, \"123\")\n\t\thelloMap[s] = \"123\"\n\t}\n}\n\nfunc Render(s string) string {\n\tif s == \"map\" {\n\t\treturn helloMap[\"100\"]\n\t}\n\tres, _ := hello.Get(\"100\")\n\treturn res.(string)\n}\n"
          }
        ]
      },
      "deposit": ""
    }
  ],
  "fee": {
    "gas_wanted": "100000000",
    "gas_fee": "1ugnot"
  },
  "signatures": [
    {
      "pub_key": {
        "@type": "/tm.PubKeySecp256k1",
        "value": "A+FhNtsXHjLfSJk1lB8FbiL4mGPjc50Kt81J7EKDnJ2y"
      },
      "signature": "6SJczVB2K9f+WOMMWvPA2QVhvQQVroNaPV1D30QtY5RctrwaW5IWxmQrbmU/+eGq0GTnoKv0VjmBr7n8UDxk+w=="
    }
  ],
  "memo": ""
}

notice that there is a @type field in some json structures. this is partly why amino json exists in the first place: so that this information can be marshaled and unmarshaled correctly, and we can find again the adequate types when we unmarshal.

the same types are also encoded, still as strings, also when we take a look at the binary:

(if you're like me, you may be wondering why in the world a type is /vm.m_addpkg, and another one is /tm.PubKeySecp256k1. i know. it's inconsistent. maybe we'll get to fixing it before mainnet; though this will likely break everything. though... that may be a good idea for a PR ;))

coming back at the original problem. how do we fix our program?

the answer is an anonymous import on the vm package.

import _ "github.com/gnolang/gno/gno.land/pkg/sdk/vm"

$ cat result.json | go  run .
{Msgs:[{Creator:g1jg8mtutu9khhfwc4nxmuhcpftf0pajdhfvsqf5 Package:0xc0002e7200 Deposit:}] Fee:{GasWanted:100000000 GasFee:1ugnot} Signatures:[{PubKey:gpub1pgfj7ard9eg82cjtv4u4xetrwqer2dntxyfzxz3pq0skzdkmzu0r9h6gny6eg8c9dc303xrrudee6z4he4y7cs5rnjwmyf40yaj Signature:[233 34 92 205 80 118 43 215 254 88 227 12 90 243 192 217 5 97 189 4 21 174 131 90 61 93 67 223 68 45 99 148 92 182 188 26 91 146 22 198 100 43 110 101 63 249 225 170 208 100 231 160 171 244 86 57 129 175 185 252 80 60 100 251]}] Memo:}

the import has a call to amino.RegisterPackage as a side-effect; which will register our types and make our program work; as our types can now be decoded into their corresponding data structures.

the source code for the "fixed" utility is here.

how do i know which package i need to import?

the vm package is not always the only one that needs to be imported; though it does import a lot of tendermint packages, so other imports are often unnecessary.

but you can search in the code for where amino registers its "packages" to try and trace back what you need to import. here's a handy ripgrep command that can help you list all of the places we use amino.RegisterPackage:

rg -t go -U '^\w.*amino\.RegisterPackage\((.*|\n)*^\)'

by looking at the second parameter in the call to "NewPackage", you can find the "namespace" that is being used; ie. what appears in front of the type name in the @type string; and in our original error message.