Convert CSV to DynamoDB JSON in JavaScript: Types and Batch Write Format
- Authors

- Name
- Geeks Kai
- @KaiGeeks

DynamoDB JSON, ordinary JSON, and a BatchWriteItem request are different formats. For the AWS CLI's --request-items argument, group operations under the table name and wrap each item in PutRequest. The array of Item objects in an S3 DynamoDB JSON import is not that request shape.
AWS also supports importing CSV from S3 into a new table; converting first is not universally required. This guide instead prepares a small, explicitly typed batch for an existing table. Conversion is offline. The later AWS write is a separate operation that can replace existing items.
This example assumes a table with id as its string partition key and no sort key. Adapt and validate both key fields before using it with a composite-key table.
| CSV field | DynamoDB attribute | Example policy |
|---|---|---|
| id | S | Required, non-blank, at most 2048 UTF-8 bytes; preserve original text |
| amount | N | Signed integer text with at most 38 digits; no decimal or exponent notation in this example |
| active | BOOL | Accept exactly lowercase true or false; reject other text |
| note | S or NULL | An empty cell becomes NULL; otherwise preserve the text, with a 1024-byte example limit |
DynamoDB represents N values as strings in the API, while BOOL and NULL carry JSON booleans. Keep amount as text throughout parsing: 9007199254740993 cannot be represented exactly by JavaScript Number. This converter intentionally accepts a narrower integer-only subset of DynamoDB numbers. Supporting decimal amounts or scientific notation requires a separately validated decimal policy; do not quietly round them through Number.
Use Node.js 22 or newer in a new, empty scratch directory. Initialize the project, record csv-parse version 6.1.0 as an exact dependency in package.json, and install it. Keep the generated package-lock.json with the example when sharing it. Automatic type casting stays off.
npm init -y
npm pkg set dependencies.csv-parse=6.1.0
npm install
Save the following as convert.cjs. It validates all rows before writing, rejects duplicate keys within the batch, accepts 1-25 rows, and refuses to replace an existing output file. It does not access AWS credentials or make a network request.
const { readFileSync, writeFileSync } = require("node:fs")
const { parse } = require("csv-parse/sync")
const REQUIRED_HEADERS = ["id", "amount", "active", "note"]
const MAX_BATCH_OPERATIONS = 25
const MAX_PARTITION_KEY_BYTES = 2048
const MAX_NOTE_BYTES = 1024
const INTEGER_TEXT = /^-?(?:0|[1-9]\d{0,37})$/
const TABLE_NAME = /^[A-Za-z0-9_.-]{3,255}$/
function validateHeaders(headerNames) {
const hasExpectedHeaders = headerNames.length === REQUIRED_HEADERS.length &&
REQUIRED_HEADERS.every(name => headerNames.includes(name))
if (!hasExpectedHeaders) {
throw new Error("Expected exactly these CSV headers: " + REQUIRED_HEADERS.join(","))
}
return headerNames
}
function toAttributeMap(record) {
if (record.id.trim() === "" || Buffer.byteLength(record.id, "utf8") > MAX_PARTITION_KEY_BYTES) {
throw new Error("id must be a non-blank partition key of at most 2048 UTF-8 bytes")
}
if (!INTEGER_TEXT.test(record.amount)) {
throw new Error("amount must be integer text with at most 38 digits")
}
if (record.active !== "true" && record.active !== "false") {
throw new Error("active must be exactly true or false")
}
if (Buffer.byteLength(record.note, "utf8") > MAX_NOTE_BYTES) {
throw new Error("note exceeds this example's 1024-byte limit")
}
return {
id: { S: record.id },
amount: { N: record.amount },
active: { BOOL: record.active === "true" },
note: record.note === "" ? { NULL: true } : { S: record.note },
}
}
function convertCsv({ csvText, tableName }) {
if (typeof tableName !== "string" || !TABLE_NAME.test(tableName)) {
throw new Error("Provide a table name of 3-255 letters, digits, underscores, dots, or hyphens")
}
const records = parse(csvText, {
columns: validateHeaders, bom: true, skip_empty_lines: true, cast: false,
})
if (records.length === 0 || records.length > MAX_BATCH_OPERATIONS) {
throw new Error("This small-batch converter accepts 1-25 records")
}
const seenKeys = new Set()
const writeRequests = records.map(record => {
const attributes = toAttributeMap(record)
if (seenKeys.has(record.id)) {
throw new Error("Duplicate partition key in this batch: " + record.id)
}
seenKeys.add(record.id)
return { PutRequest: { Item: attributes } }
})
return { [tableName]: writeRequests }
}
function writeConversion({ inputPath, outputPath, tableName }) {
const csvText = readFileSync(inputPath, "utf8")
const requestItems = convertCsv({ csvText, tableName })
// Refuse an existing output; the conversion itself makes no AWS request.
writeFileSync(outputPath, JSON.stringify(requestItems, null, 2) + "\n", { flag: "wx" })
}
module.exports = { convertCsv, writeConversion }
if (require.main === module) {
const [, , inputPath, outputPath, tableName] = process.argv
if (!inputPath || !outputPath || !tableName) {
console.error("Usage: node convert.cjs input.csv output.json ExampleTable")
process.exitCode = 1
} else {
try {
writeConversion({ inputPath, outputPath, tableName })
console.log("Created request-items JSON. No AWS write was performed.")
} catch (error) {
console.error("CSV conversion failed:", error)
process.exitCode = 1
}
}
}
The header validator accepts the four named columns in any order, but rejects missing, extra, or duplicate headers. The CSV parser handles quoted commas, escaped double quotes, a UTF-8 BOM, and record-length errors; splitting each line on a comma would not.
Save input.csv:
id,amount,active,note
a1,9007199254740993,true,"Hello, CSV"
a2,0,false,
Then run:
node convert.cjs input.csv output.json ExampleTable
The resulting output.json is exactly the map required by --request-items:
{
"ExampleTable": [
{
"PutRequest": {
"Item": {
"id": {
"S": "a1"
},
"amount": {
"N": "9007199254740993"
},
"active": {
"BOOL": true
},
"note": {
"S": "Hello, CSV"
}
}
}
},
{
"PutRequest": {
"Item": {
"id": {
"S": "a2"
},
"amount": {
"N": "0"
},
"active": {
"BOOL": false
},
"note": {
"NULL": true
}
}
}
}
]
}
The large integer remains unchanged, false is a boolean rather than a truthy string, and the blank note follows the explicit NULL policy. A blank amount, a value such as yes for active, a duplicate id, or 26 rows fails conversion. Change the input rather than accepting a silently guessed type. A file with no records also fails.
Confirm the AWS account, region, table name, and key schema separately. PutRequest replaces an existing item with the same complete primary key. BatchWriteItem cannot apply per-item conditions, so this command is inappropriate when an existing item must be protected from replacement. Use a conditional PutItem workflow in that case.
If the intended table and replacement behavior have been verified, the separate CLI operation is:
aws dynamodb batch-write-item --request-items file://output.json
Inspect the response for UnprocessedItems. A successful HTTP response does not mean every requested write completed. Retry only the returned unprocessed operations, with backoff, until they are empty or your retry limit is reached. A batch is not an all-or-nothing transaction. Do not resend the entire original batch blindly.
The API permits at most 25 operations and a 16 MB request, with a 400 KB stored-item limit. The example's small field limits keep its generated records small; for a different schema or bulk importer, validate item and request sizes, split batches, and handle capacity and retry behavior explicitly. The offline conversion does not verify IAM permissions, actual table schema, capacity, or service-side writes.
| Consumer | Expected shape |
|---|---|
| AWS CLI --request-items | A map from table name to PutRequest or DeleteRequest operations, as generated above |
| Complete BatchWriteItem API request | The same map nested inside the RequestItems property |
| S3 import with DynamoDB JSON | One JSON object per line, each wrapping an Item; not the CLI batch map |
| S3 import with CSV | A supported CSV import into a new table; non-key column typing follows the documented import rules |