Option to reduce bundle size by removing some languages from "Code Block" feature?

[sindras]

Is your feature request related to a problem? Please describe.

Follow-up from #1177, is there a way to reduce bundle size by opting out from some languages?
Even though the languages have been filtered out as per discussion in the PR, they are still bundled in the production assets.
Screenshot attached below.

Also there does not seem to be any documentation about configuring the code block in the docs website.

Describe the solution you'd like

1. Provide the option to configure the Code Block, especially supported languages.
2. Tree-shake the unused languages.
3. Update the documentation to include an example of configuring the Code Block.

Additional context

This is generated by running npm run build with Remix framework.

Bonus

I am willing to help if someone can point me in the right direction regarding the tree-shaking.
Thank you so much for creating BlockNote! ❤️

[YousefED]

Unfortunately #1216 doesn't solve this, but only makes sure the languages are not part of the @blocknote/core bundle. Your application would still have the language files in the output.

Note that for the end-user, this should not be an issue as languages are loaded dynamically

However, I understand you might still want to reduce your deployment output.

To do this, seems like there are two options:
a) we need to update the code to use https://shiki.style/guide/bundles#fine-grained-bundle and make languages configurable differently (requires work on BlockNote side)
b) possibly, remove languages manually using your rollup / webpack config

[sindras]

Note that for the end-user, this should not be an issue as languages are loaded dynamically

Yes I noticed that the lang is loaded dynamically.

However, I understand you might still want to reduce your deployment output.

Exactly, these assets are significant in terms of size. Our deployment infrastructure charged us to a higher-tier usage plan based on upload size. We could have saved budget if we can reduce this dependencies. Thanks for taking a look at this, though 👍 . I will consider the options.

[areknawo]

@sindras If you're using a Rollup-based bundler (e.g. Vite), you can specify the languages to exclude as external:

...
rollupOptions: {
  external: (source) => {
    // Exclude JavaScript
    if(source.includes("shiki/dist/langs/javascript")) {
      return true;
    }
    
    return false;
  }
}
...

Alternatively you should be able to externalize all shiki/dist/langs by default and only include the ones you need.

You'll also have to use customizeCodeBlock() to specify the languages that you've bundled and that should be included in the select menu.

Before you use this method though, wait for #1219 to be merged, as it fixes a separate issue that was duplicating the languages in the output bundle (potentially breaking the method above).

[sindras]

That's super helpful, thank you @areknawo ! I will keep an eye on the next release 👍

Just curious, is this the right way to customize the CodeBlock? I am having a TS error because the default language values are coming as string[] of lang ids but the customization's option seems to expect a SupportedLanguageConfig[] (id, name, match) array.

Perhaps the CodeBlockOptions supportedLanguages should be of type SupportedLanguageConfig["id"][]? So users can just pass the language IDs from Shiki doc instead of a language config object.

import {
  BlockNoteSchema,
  defaultBlockSpecs,
  customizeCodeBlock,
  defaultCodeBlockPropSchema,
} from '@blocknote/core';

const schema = BlockNoteSchema.create({
  blockSpecs: {
    ...defaultBlockSpecs,
    codeBlock: customizeCodeBlock({
      defaultLanguage: 'javascript',
      indentLineWithTab: true,

      // This currently gives TS error
      supportedLanguages: defaultCodeBlockPropSchema.language.values.filter(id => ['javascript', 'html'].includes(id)),
    }),
  },
});

function MyEditor() {
 const editor = useCreateBlockNote({schema});
  // ... return <BlockNoteView {...} />
}

[sindras]

Another question, do we support switching themes of the CodeBlock? If not, we probably should exclude them from build as well

[areknawo]

@sindras My idea behind SupportedLanguageConfig was to allow different aliases to be assigned to specific languages (which already turned out helpful to support VS Code's typescriptreact alias for Shiki's tsx).

Here's how you would configure the CodeBlock:

customizeCodeBlock({
  defaultLanguage: "javascript",
  supportedLanguages: [
    {
      id: "javascript",
      match: ["javascript", "js"],
      name: "JavaScript",
    },
  ],
});

That said, I see how accepting a list of IDs could make things easier. Perhaps it'd be best to support an array of SupportedLanguageConfig | string?

As for the syntax highlighting themes - we could support them or exclude unused themes by default. That said, per earlier comment, the latter option wouldn't change much for you as the end-user of the package and you will likely still need to externalize them in your bundler.

@YousefED Let me know if you have any feedback

[sindras]

That makes sense, either with SupportedLanguageConfig or with string ID works for me. However, I imagine some developers would start with the default set of languages, and then filter out few languages that they do not want to support - instead of the other way around. Having the default set of languages exported to support that would be great, but I understand it is not a must-have.

Thank you for your support to provide code block in this editor! This is a big, cool feature that convinces me to use BlockNote 😄