Typefusion allows you to run TypeScript scripts and materialize the results into a database. It enables you to create complex workflows by allowing scripts to reference each other's results. Inspired by Data Build Tool (DBT).
To begin using Typefusion, follow these steps:
Install Typefusion using your preferred package manager:
npm install typefusion
# or
pnpm install typefusion
# or
yarn add typefusion
# or
bun add typefusion
Configure your database connection using one of these methods (PostgreSQL, MySQL, and Clickhouse are supported):
PG_DATABASE_URL or MYSQL_DATABASE_URL or CLICKHOUSE_DATABASE_URL (and optionally CLICKHOUSE_USER, CLICKHOUSE_PASSWORD, and CLICKHOUSE_DATABASE if not using the defaults) environment variable.PG_DATABASE, PG_HOST, PG_PORT, PG_PASSWORD, and PG_USER (for postgres) or MYSQL_DATABASE, MYSQL_HOST, MYSQL_PORT, MYSQL_PASSWORD, and MYSQL_USER (for mysql) or CLICKHOUSE_DATABASE, CLICKHOUSE_HOST, CLICKHOUSE_PORT, CLICKHOUSE_PASSWORD, and CLICKHOUSE_USER (for clickhouse).Create a directory for your scripts (e.g., workflows).
Write your TypeScript scripts in the created directory.
Compile your TypeScript scripts into your desired output directory (e.g. scripts). Typefusion runs on the compiled JavaScript output, so ensure you don't bundle your scripts to preserve the directory layout. Avoid outputting to directories like dist or build (see Troubleshooting)
After following the above instructions, create a script file in the directory, for example, main.ts:
// or mySqlType for mysql or clickhouseType for clickhouse
import { pgType, typefusionRef, TypefusionDbScript } from "typefusion";
const mainSchema = {
id: pgType.integer().notNull(),
name: pgType.text().notNull(),
age: pgType.integer().notNull(),
email: pgType.text().notNull(),
address: pgType.text().notNull(),
};
export default {
name: "main",
schema: mainSchema,
resultDatabase: "postgresql",
run: async () => {
console.log("running main");
return {
data: [
{
id: 1,
name: "John Doe",
age: 30,
email: "john.doe@example.com",
address: "123 Main St",
},
],
};
},
} satisfies TypefusionDbScript<typeof mainSchema>;
Warning: Typefusion is native ESM and does not provide a CommonJS export.
Typefusion can be used in two primary modes: CLI and library. Both modes offer similar functionality, allowing you to choose the most suitable approach for your project.
The Typefusion CLI provides a convenient way to run your scripts from the command line. To use the CLI:
Add a script to your package.json:
"scripts": {
"run-typefusion": "dotenv -- typefusion ./scripts"
}
This example assumes you're using dotenv for environment variable management and that your compiled scripts are in the ./scripts directory.
Run your scripts:
npm run run-typefusion
To explore all CLI options, run:
npm run typefusion --help
You can also use Typefusion as a library in your TypeScript projects:
import typefusion from "typefusion";
await typefusion({
directory: "./scripts",
});
This approach allows for more programmatic control and integration with your existing TypeScript applications.
Typefusion Refs enable you to reference the results of one script in another, facilitating the creation of complex workflows. Here's an example:
// or mySqlType for mysql
import { pgType, typefusionRef, TypefusionDbScript } from "typefusion";
import main from "./main.js";
const smallSchema = {
small: pgType.text().notNull(),
};
export default {
name: "typefusion_ref",
schema: smallSchema,
resultDatabase: "postgresql",
run: async () => {
const result = await typefusionRef(main);
console.log("typefusion ref main result", result);
return {
data: [
{
small: "smallString" as const,
},
],
};
},
} satisfies TypefusionDbScript<typeof smallSchema>;
For cases where you only need the table name without fetching the full data, use the typefusionRefTableName function:
import { typefusionRefTableName } from "typefusion";
import main from "./main.js";
const tableName = await typefusionRefTableName(main);
console.log("typefusion ref table name", tableName); // Outputs: "main"
To help you get started, we've provided several examples:
Find more examples in the examples directory.
For detailed API documentation, visit the reference docs. The documentation is generated from JSDoc comments in the source code, providing comprehensive information on usage and features.
Typefusion is built with Effect. Refer to the reference docs for details on Effect-suffixed functions and their usage. Most of the time, you just need to suffix the types and functions with Effect to use them. For example, typefusionRef becomes typefusionRefEffect, TypefusionDbScript becomes TypefusionDbScriptEffect, and run becomes runEffect. The typefusion library export is typefusionEffect.
This can happen because the library Typefusion depends on to generate the dependency graph ignores common build directories by default. For example, you cannot output scripts to build, dist, and several others (full list here).
If you encounter any other issues while using Typefusion:
Contributions to Typefusion are welcome! To contribute:
main.If it's a larger change, please open an issue to discuss your proposed changes or feature additions.