Why Files.readString breaks the file your colleague saved on Windows

A colleague exports a CSV from Excel, or saves a JSON file in Notepad as "ANSI", and commits it to the integration repo. Your code calls Files.readString(path) and hands the text to the parser. The Linux pipeline fails with java.nio.charset.MalformedInputException: Input length = 1. Nobody touched the content, and the file's diff is empty.

People usually blame the Linux CI box first, but Linux has nothing to do with it. According to the javadoc, Files.readString(Path) decodes bytes as UTF-8 on every operating system. Your colleague's file isn't UTF-8.

What follows: a repro using a one-byte file, why JEP 400 doesn't change this method, why -Dfile.encoding doesn't either, and a fix in two parts. The files you write yourself get an explicit UTF-8 contract. The file that arrives from the partner gets a declared charset.

Environment

• Temurin 25.0.4 (25.0.4+7-LTS), no --enable-preview
• Maven 3.9.12, maven.compiler.release=25
• maven-compiler-plugin 3.14.0, maven-surefire-plugin 3.6.0
• org.junit.jupiter:junit-jupiter 6.1.3
• Linux, platform encoding: UTF-8, no Spring

The pom.xml, trimmed:

<groupId>devdojo</groupId>
<artifactId>files-readstring-charset-fixture</artifactId>
<version>1.0.0-SNAPSHOT</version>
<properties>
  <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  <maven.compiler.release>25</maven.compiler.release>
  <junit.version>6.1.3</junit.version>
</properties>
<dependencies>
  <dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter</artifactId>
    <version>${junit.version}</version>
    <scope>test</scope>
  </dependency>
</dependencies>
<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-compiler-plugin</artifactId>
      <version>3.14.0</version>
      <configuration>
        <release>25</release>
      </configuration>
    </plugin>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>3.6.0</version>
    </plugin>
  </plugins>
</build>

project.build.sourceEncoding=UTF-8 tells the compiler how to read your .java files. It has no effect on how readString decodes the partner's CSV. The property turns up in plenty of encoding "fixes" copied off the internet, and it does nothing for this problem.

One byte, one file, and readString without a charset

You don't need a CSV or a JSON file to reproduce this. The byte 0xE9 is enough:

• In windows-1252 and ISO-8859-1, 0xE9 is é (U+00E9).
• In UTF-8, a lone 0xE9 is a malformed sequence. It opens a three-byte sequence, and the other two bytes never show up. In UTF-8, é takes two bytes: C3 A9.

A name like "José" saved as ANSI ends up as exactly this on disk: the accented letter is a single byte that isn't valid UTF-8.

The Files javadoc for JDK 25 says Files.readString(Path) reads the content and decodes the bytes using the UTF-8 charset. It doesn't check Charset.defaultCharset() and it ignores -Dfile.encoding. If you want a different charset, the overload is Files.readString(Path, Charset). Files.newBufferedReader(Path) works the same way: the javadoc says UTF-8, and the overload is newBufferedReader(Path, Charset).

The project's Demo class writes {(byte) 0xE9} to a temp file and reads it back several ways. Here's the output on this host with the JVM default (UTF-8):

bytes=[-23]
Charset.defaultCharset()=UTF-8
new String(bytes, defaultCharset)=�
new String(bytes, ISO-8859-1)=é
Files.readString(path) threw java.nio.charset.MalformedInputException: Input length = 1
Files.readString(path, windows-1252)=é
Files.readString(path, ISO-8859-1)=é
Files.newBufferedReader(path) threw java.nio.charset.MalformedInputException: Input length = 1
newBufferedReader(path, windows-1252)=é

The same byte gets two different reactions. There's only one file. The difference is the decoder's policy:

• Files.readString and Files.newBufferedReader report malformed input and throw MalformedInputException.
• new String(bytes, UTF-8) replaces malformed input with U+FFFD (�) and moves on.

The exception is the more honest of the two. The � doesn't break anything right away. It turns up a few weeks later, in a report listing a customer called "Jos�".

JEP 400 changed the JVM default, not this method

JEP 400 shipped in Java 18 and still applies in JDK 25. It made UTF-8 the default charset for the APIs that depend on the default. The point was to make new String(byte[]), InputStreamReader without a charset, FileReader and friends behave the same on every operating system.

The no-charset methods on Files aren't part of that group. Their javadoc already said UTF-8 before JEP 400, so the JEP didn't "fix" them and didn't "break" them either. That's also why changing -Dfile.encoding doesn't help. Here's what Demo produced when run with -Dfile.encoding=ISO-8859-1 and with -Dfile.encoding=windows-1252:

In all three runs, native.encoding stayed UTF-8. file.encoding and Charset.defaultCharset() followed the flag, and Files.readString(path) threw the same exception all three times.

So adding -Dfile.encoding=windows-1252 to the CI's MAVEN_OPTS while keeping readString(path) fixes nothing. It changes how some new String elsewhere in the codebase decodes, leaves readString exactly as it was, and makes the build depend on a flag that nobody will remember the reason for.

The Charset documentation explains the default charset. The InputStreamReader documentation shows that the no-charset constructor uses that default. Those contracts are different from the contract of Files.readString(Path), and that's why the same file behaves differently in each API.

The test that writes 0xE9 into @TempDir

Don't commit a "broken" .csv to the repo. Editors, .gitattributes and merge tools all change encodings without telling anyone, and the test would end up testing whatever editor the last committer used. This test writes the bytes straight into @TempDir, with no String involved:

package com.devdojo.encoding;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.nio.charset.Charset;
import java.nio.charset.MalformedInputException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Arrays;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;

class ReadStringCharsetTest {

    private static final byte[] LATIN1_E_ACUTE = {(byte) 0xE9};

    private static Path latin1File(Path dir) throws IOException {
        Path p = dir.resolve("colega-windows.csv");
        Files.write(p, LATIN1_E_ACUTE);
        return p;
    }

    @Test
    void readStringWithoutCharsetThrowsMalformedInputException(@TempDir Path dir) throws IOException {
        Path p = latin1File(dir);
        assertEquals("[-23]", Arrays.toString(Files.readAllBytes(p)));
        MalformedInputException ex = assertThrows(
                MalformedInputException.class,
                () -> Files.readString(p));
        assertTrue(ex.getMessage() != null && ex.getMessage().contains("Input length = 1"),
                () -> "unexpected message: " + ex.getMessage());
    }

    @Test
    void readStringWithWindows1252AndIso88591ReturnsEAcute(@TempDir Path dir) throws IOException {
        Path p = latin1File(dir);
        assertEquals("é", Files.readString(p, Charset.forName("windows-1252")));
        assertEquals("é", Files.readString(p, Charset.forName("ISO-8859-1")));
        assertEquals("é", Files.readString(p, StandardCharsets.ISO_8859_1));
        assertEquals("é", PartnerFiles.readWindows1252(p));
        assertEquals("é", PartnerFiles.readString(p, Charset.forName("windows-1252")));
    }

    @Test
    void newBufferedReaderWithoutCharsetAlsoUtf8OnJdk25(@TempDir Path dir) throws IOException {
        Path p = latin1File(dir);
        MalformedInputException ex = assertThrows(MalformedInputException.class, () -> {
            try (BufferedReader r = Files.newBufferedReader(p)) {
                r.readLine();
            }
        });
        assertTrue(ex.getMessage() != null && ex.getMessage().contains("Input length = 1"));
        try (BufferedReader r = Files.newBufferedReader(p, Charset.forName("windows-1252"))) {
            assertEquals("é", r.readLine());
        }
        try (BufferedReader r = new BufferedReader(
                new InputStreamReader(Files.newInputStream(p), Charset.forName("windows-1252")))) {
            assertEquals("é", r.readLine());
        }
    }

    @Test
    void defaultCharsetStringCtorIsNotTheSameContractAsReadString(@TempDir Path dir) throws IOException {
        Path p = latin1File(dir);
        String viaDefault = new String(Files.readAllBytes(p), Charset.defaultCharset());
        assertEquals("UTF-8", Charset.defaultCharset().name());
        assertEquals("\uFFFD", viaDefault, "UTF-8 decode of 0xE9 yields U+FFFD, readString throws instead");
        assertThrows(MalformedInputException.class, () -> Files.readString(p));
    }
}

mvn -q test

Result on this host: ReadStringCharsetTest with Tests run: 4, Failures: 0, Errors: 0, Skipped: 0.

The last test pins the default to UTF-8 on purpose, because UTF-8 has been the JDK default since JEP 400. If someone slips -Dfile.encoding into Surefire, that test fails, and it should. It's there to document that the build has started depending on a flag.

Your files in UTF-8, partner files in windows-1252

The fix depends on who writes the file.

If the file is yours (your service writes it and your service reads it), the contract is UTF-8, and you write it into the code on both sides:

Files.writeString(path, conteudo, StandardCharsets.UTF_8);
String lido = Files.readString(path, StandardCharsets.UTF_8);

In practice, readString(path, UTF_8) does the same thing as readString(path). The difference is that the charset shows up in the code. Someone reading that line a year from now can see the contract is UTF-8 without opening the javadoc.

If the file comes from outside (a partner, an exported spreadsheet, "save as ANSI"), don't call Files.readString(Path) without a charset. Declare the charset at a single read point:

package com.devdojo.encoding;

import java.io.IOException;
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Path;

/**
 * Inbound partner files (CSV/JSON saved as Windows-1252 / ISO-8859-1) must
 * never go through {@link Files#readString(Path)} — that overload is UTF-8.
 */
public final class PartnerFiles {

    private PartnerFiles() {}

    public static String readString(Path path, Charset charset) throws IOException {
        return Files.readString(path, charset);
    }

    public static String readWindows1252(Path path) throws IOException {
        return readString(path, Charset.forName("windows-1252"));
    }
}

If the encoding varies from partner to partner, make it configuration: PartnerFiles.readString(path, Charset.forName(configEncoding)). The details that tend to catch people:

• windows-1252 is not in StandardCharsets. Use Charset.forName("windows-1252"). The constant that does exist is StandardCharsets.ISO_8859_1.
• For 0xE9, windows-1252 and ISO-8859-1 both give you é, and the test confirms it. They differ in the 0x80–0x9F range, where windows-1252 puts printable characters such as curly quotes and €. For "ANSI" files coming from Windows, windows-1252 is the right name.
• The exception message is just Input length = 1. It has no path and no expected charset, which doesn't help anyone in a job processing forty files. At the read point, catch CharacterCodingException (the superclass of MalformedInputException) and log the path and the charset you used before rethrowing. Then when CI fails, the log tells you which file arrived in which encoding.

What DevDojo recommends: in production code, no read of an external file goes through Files.readString(Path) or Files.newBufferedReader(Path) without a charset, and internal files pass StandardCharsets.UTF_8 explicitly on both write and read. A helper with a fixed charset stops being enough once the same partner sends different encodings from one delivery to the next. At that point you have a contract problem with the partner: agree on the encoding in the integration agreement, or ask for UTF-8. Guessing the charset from the bytes is a separate topic, and it isn't covered here.

What's out of scope and next step

This covers files on disk read through NIO. HTTP response charsets, Content-Type and Spring clients follow a different contract. InputStreamReader without a charset, like FileReader, follows the JVM default. It's a similar trap with a different rule: there, -Dfile.encoding does change the result, as the table showed.

To check your own project, look for file reads with no charset. The grep below is approximate and misses calls with a comma inside the argument, but it catches most of them:

grep -rnE 'Files\.(readString|newBufferedReader)\([^,]*\)' src/main/java

For every match that reads someone else's file, pass the charset explicitly and write a test that writes 0xE9 into @TempDir, like ReadStringCharsetTest does. Then run mvn -q test. To see for yourself that the JVM flag doesn't help, run java -Dfile.encoding=ISO-8859-1 -cp target/classes com.devdojo.encoding.Demo. Files.readString(path) throws the same MalformedInputException, two lines below Charset.defaultCharset() reporting ISO-8859-1.