Skip to content
All notes

4 min read

GridFS from Bun, without the ceremony

Why I wrote bun-gridfs-storage, what the old Multer engine got wrong under Bun, and the four decisions that made file uploads boring again.

Every product I have shipped in the last four years stores files somewhere, and every one of them started with the same conversation: object storage, or the database we already run? For Keforo the answer was MongoDB. The files were small, they belonged to documents that already lived there, and one backup covered everything. That decision was easy. Getting uploads to work under Bun was not.

#What broke

The Node ecosystem has an answer for this: multer parses the multipart body, and a storage engine decides where each file goes. multer-gridfs-storage was that engine for MongoDB, and it had years of production behind it.

Under Bun it fell over in two places. Its stream handling assumed Node internals that Bun's implementation does not honour, and its EventEmitter wiring emitted before listeners were attached. Uploads hung, or resolved with a file record that pointed at nothing. The package had not seen a release in a long time, and I did not want a fork with patches nobody else would run.

So I wrote a storage engine that does exactly one job and does it the same way on both runtimes. It became bun-gridfs-storage.

#What a storage engine actually has to do

Multer's contract is small. An engine receives the request and the incoming file stream, writes it somewhere, and calls back with whatever should end up on req.file. For GridFS that means opening an upload stream on a bucket and piping into it.

upload.tstypescript
import { BunGridFSStorage } from 'bun-gridfs-storage'
import multer from 'multer'
import mongoose from 'mongoose'

await mongoose.connect(process.env.MONGO_URL!)

const storage = new BunGridFSStorage({
  db: mongoose.connection.db,
  file: (req, file) => ({
    filename: `${Date.now()}-${file.originalname}`,
    bucketName: 'uploads',
  }),
})

const upload = multer({ storage, limits: { fileSize: 5 * 1024 * 1024 } })

app.post('/upload', upload.single('file'), (req, res) => {
  res.json({ file: req.file })
})

Everything else in the package exists to make that boring in production. Four decisions did most of the work.

#1. The database can arrive late

The storage engine is usually constructed at module load, long before the connection is open. Instead of forcing a boot order, db accepts either a Db or a Promise<Db>. Uploads that arrive before the promise settles wait for it.

typescript
const storage = new BunGridFSStorage({
  db: getDb(),
  file: (req, file) => ({ filename: file.originalname, bucketName: 'uploads' }),
})

async function getDb() {
  if (mongoose.connection.readyState === 1) return mongoose.connection.db
  return new Promise((resolve, reject) => {
    mongoose.connection.once('open', () => resolve(mongoose.connection.db))
    mongoose.connection.once('error', reject)
  })
}

#2. One callback decides everything about a file

Filename, bucket, chunk size, content type and metadata all come from the same file callback, and it may be async. That is where a real application puts the user id, the tenant, or a content hash it computed elsewhere.

typescript
const storage = new BunGridFSStorage({
  db: mongoose.connection.db,
  file: async (req, file) => ({
    filename: `${crypto.randomUUID()}-${file.originalname}`,
    bucketName: 'uploads',
    chunkSize: 255 * 1024,
    contentType: file.mimetype,
    metadata: {
      userId: req.user.id,
      originalName: file.originalname,
    },
  }),
})

#3. Events, not logs

The engine extends EventEmitter and emits connection, file, streamError and connectionFailed. That sounds like a small thing until you need to count failed uploads per tenant, or start a virus scan the moment a file lands, without touching the route handler.

typescript
storage.on('file', (file) => metrics.increment('uploads', { bucket: file.bucketName }))
storage.on('streamError', (error, config) => log.error({ error, config }, 'upload failed'))

#4. Bun first, Node still

The tests run under bun test, and the build ships CommonJS and ESM. Node 18 and later works unchanged. I do not want two file-upload stories in one codebase because a service happens to run on a different runtime.

#Reading it back

Downloads and deletes go straight through the driver's bucket. The engine exposes it so you never construct a second one.

typescript
app.get('/files/:name', (req, res) => {
  const bucket = storage.getBucket()
  if (!bucket) return res.status(503).send('storage not ready')
  bucket
    .openDownloadStreamByName(req.params.name)
    .on('error', () => res.status(404).end())
    .pipe(res)
})

#When GridFS is the wrong answer

If your files are large, public, or served to many readers, a CDN in front of object storage will beat this every time, and it will be cheaper. GridFS earns its place when files are private, modest in size, tied to documents you already query, and you want one backup, one access-control model and one connection string. That was every Keforo upload, which is why the package exists.

Did this land?

Conversation

Building something like this?

Tell me what you are working on. I reply within a day.

Write to me