Converted research paper to Kindle EPUB

Summary

Closed, Completed

Understand
Classified source PDF and inspected intermediate Markdown.
Understand
Build
Created and improved a custom build script with state-tracking and table handling.
Build
Build
Built the EPUB file.
Build
Verify
Performed deterministic QA checks and readability spot-checks.
Verify
Verify
Verified EPUB TOC structure and heading correctness.
Verify

Convert this research paper to read on kindle paperwhite 7th gen. /Users/kiran/Downloads/DeepSeek_V4.pdf

ReadREADME.md

ReadCONVERSION_QA_CHECKLIST.md

Source classified as: text-based PDF (LaTeX-generated, 58 pages, single column). Running first pass with --keep-md and --no-pause so I can inspect the intermediate Markdown and rebuild after structural cleanup.

ReadDeepSeek_V4.md130 lines

ReadDeepSeek_V4.md280 lines

I'm seeing that the table of contents and figure captions are cluttering the output, so I need to clean this up by removing the redundant TOC from the body since it's already in the metadata, stripping out figure labels and axis annotations, using the paper's actual structure to rebuild the headings properly, and making sure to escape any HTML-like syntax that might cause parsing issues.

Given the complexity of a 58-page LaTeX-generated PDF, I'm weighing two approaches: manually reconstructing the semantics using the paper's TOC structure, or applying targeted transformations to the extracted text. Since the extraction is mostly solid with just specific problem areas, I think the better move is to re-extract the raw text and build a custom reconstruction script that uses the paper's TOC as a scaffold to rebuild the markdown cleanly, stripping out the TOC pages and figure noise along the way.

I'm settling on a concrete approach now. I'll create a Python script that processes the raw PDF text output to build clean markdown—using pdftotext without layout preservation to maintain proper reading order, then skipping the table of contents pages and starting from the introduction section, with regex patterns to detect and format major section headings appropriately.

The script will also handle cleanup tasks like removing page numbers, reflowing wrapped paragraphs, fixing hyphenated words split across lines, escaping XML-like tags that might confuse pandoc, stripping repeated headers and footers, and carefully removing orphaned figure caption fragments while preserving actual captions. I need to verify exactly how the section numbers and titles are laid out in the raw text before finalizing the regex patterns.

I'm seeing the document structure now—the body content starts around line 494 with the introduction, and everything before that is front matter like the title, abstract, and table of contents. Looking at how figures are positioned, there's a consistent pattern where figure captions appear between the text and the underlying chart data. The captions themselves carry important information, so I should preserve those while filtering out the raw axis fragments and chart metadata that follow them.

I'm realizing those markup tags are literal text examples in the paper that need to be escaped in a code block so pandoc doesn't interpret them. I'm going to write a custom build script that reads the raw PDF text output and processes it carefully, starting by skipping past the title and abstract sections.

The script will walk through the document detecting heading patterns and converting them to markdown, joining wrapped lines within paragraphs, removing page numbers, and identifying code-block content like the instruction text and tool-call schemas to wrap in fenced blocks. I'll also need to escape any raw <think> tags or <|DSML|...> tokens that appear in regular prose by wrapping them in backticks. For the references section at the end, I should check whether it's worth preserving as-is since it's a standard bibliography.

Now I'm mapping out the build script structure—I'll read the raw PDF text output, skip the initial boilerplate up to the introduction, then process each line to rejoin wrapped paragraphs and detect heading patterns so I can emit proper markdown formatting while filtering out extraneous content.

I need to handle several special cases: drop standalone page numbers, preserve figure and table captions as italicized text, strip orphan label-like lines conservatively, and wrap Tables 3 and 4 in code fences to protect their XML-like structure. For the <think> tags and similar constructs scattered through the prose, I'll escape them by wrapping in backticks to prevent XML parsing issues.

References can stay mostly as-is but I'll need to rejoin any URLs that got split across lines, and the appendix author list is fine to keep. The Unicode mathematical characters like 𝑍 and 𝐻 ·𝑊 should render fine without special handling—Pandoc won't misinterpret them as TeX math.

The real issue is stray $ characters triggering math mode and causing Pandoc to choke on sequences like $...$ that contain XML-like syntax. I need to search for and escape those dollar signs.

I see that Table 4 has dollar signs in the schema that need to be wrapped in a code fence to avoid parsing issues. Table 3 looks fine with just plain instruction text. Now I'm moving on to draft the build script that will process the raw PDF text output, walking through the lines and emitting the formatted content.

I'm setting up heading detection patterns that anchor to the start of each line—single-level headings like "Introduction" and "Architecture," multi-level numbered sections, lettered appendices, and special sections like References. Page numbers that appear as standalone integers get dropped, while figure and table captions get formatted as italicized blocks.

For the problematic tables, I'm identifying them by their content boundaries—Table 3 ends before "model leverages" and Table 4 ends before "window of DeepSeek-V4"—and wrapping those ranges in code fences. Actually, I'm realizing the core issue is just escaping the special tokens like <think> and <|DSML|...> that appear in the schema, so I'll handle that with a global replacement approach instead.

Building the script now with a simpler architecture: read the raw text, skip everything before "1. Introduction," then process paragraph by paragraph. For each block, check if it's a heading pattern, a page number to skip, a figure or table caption, or regular prose that needs dehyphenation and HTML escaping.

Tables 3 and 4 get special handling by detecting their starting patterns and wrapping them in code fences. For URLs split across lines, I need to detect when a line contains http and the next line continues without a space, then join them without adding extra whitespace to preserve the URL structure.

For soft hyphens introduced by PDF wrapping, I'll use a simple heuristic: if a line ends with - and the next line starts with a lowercase letter, I'll remove the hyphen and join directly, assuming it's a word break rather than a genuine compound word.

For URLs split across lines, I need to be more careful. Rather than trying to detect every possible URL pattern upfront, I'll write a targeted regex that handles the specific broken patterns I see in the references, then do a broader cleanup pass to catch any remaining splits where a line ends with / or contains http and the next line continues with what looks like a URL fragment.

I'm noticing a pattern where URLs get split across line breaks without any special formatting—like https://artificialanalysis.ai/methodolog continuing as y/intelligence-benchmarking#gdpval-aa. on the next line. To catch this, I need to check if a line contains a URL scheme and ends with a non-space character while the next line starts with one, then join them directly.

Implementing a function that handles paragraph line joining... Refining the regex approach...

I'm realizing the regex needs to be smarter about when to stitch URLs across line breaks. The issue is distinguishing between a genuinely broken URL and a URL that's legitimately followed by new prose. If the next line starts with a capital letter, it's probably a new sentence, not a URL continuation. So I should only stitch when the fragment after the space looks like it's part of a URL—lowercase start, URL-safe characters—rather than the beginning of a new sentence. I'm realizing the issue with my regex approach — when I join lines with a space, I'm accidentally capturing the start of the next paragraph as part of the URL match. I need a better heuristic to detect when a URL is actually split across lines versus when it's genuinely followed by new content.

Looking at the actual example, the URL https://artificialanalysis.ai/methodolog ends mid-word on one line, and the next line continues with y/intelligence-benchmarking#gdpval-aa. — so I should join without a space when the previous line's URL ends without a clear terminator and the next line starts with URL-class characters like slashes or dots.

The simplest approach: when the last word of a line contains http and the next line begins with lowercase letters or URL punctuation like /, ?, #, or ., treat it as a URL continuation and join without spacing.

I'm parsing out the URL fragments to identify where each link is pointing—there's a performance matrix doc from NVIDIA, a CUDA programming guide section on kernels, and what looks like an artificial intelligence benchmarking resource.

The pattern I'm noticing is that when a URL fragment continues on the next line, it starts with lowercase characters or digits rather than a capital letter like a normal sentence would. So my heuristic is straightforward: if the previous line ends with http:// or https:// and the next line begins with a lowercase letter or digit, I should join them without a space.

Now I'm ready to write the build script that applies this logic.

Now I have enough to design a robust custom builder. Let me create a dedicated script for this paper.

I'm noticing the paragraph detection is treating "Appendix" and "A. Author List and Acknowledgment" as separate when they should be grouped together without a blank line between them. Looking at the actual input structure, these lines are part of the same section, so I need to adjust how I'm splitting paragraphs to keep them connected.

The cleanest approach is to iterate through individual lines rather than pre-split paragraphs, treating heading lines themselves as natural paragraph boundaries so I can properly separate the appendix sections.

Editedbuild_deepseek_v4.py

I'm spotting a few parsing errors in the output. There's a bogus heading at line 852, a leftover instruction fragment at line 954 where the "2." prefix got misinterpreted as a chapter marker, and another false match at line 969 where the Table 4 content triggered the heading regex because it started with ## Tools. I need to refine the pattern matching to avoid these false positives.

The core issue is that the original PDF had literal hash marks in the tool-call schema, which my code didn't escape properly. I should wrap those tables in code blocks to prevent them from being parsed as headings. For the references section, the real problem is that author names like "A. Agache" and "B. Jacob" are matching the appendix heading pattern when they shouldn't. I need to tighten the regex to only recognize actual appendix headers by constraining it to specific titles or requiring the line to be short and in title case, rather than matching any capital letter followed by a period. I'm realizing the issue — references have commas in their author lists, so I can use that to distinguish them from actual appendix headings. Let me hardcode the known appendix section titles instead of relying on regex patterns that keep matching reference lines.

Readdeepseek-v4.md35 lines

I see the issue now — the PDF table extraction pulled the column header "# Shots" into the text as a separate line, and my heading detection regex mistakenly tried to parse it as a markdown heading, but the pattern doesn't actually match that format.

Looking back at the source, "# Shots" is literally what pdftotext extracted, and since it starts with #, it's being rendered as a markdown heading when it shouldn't be. I need to escape leading # characters in paragraphs that aren't actual headings.

The real problem is that Tables 1 through 8 are getting mangled during extraction — the dense numeric data and column alignment are collapsing into fragmented paragraphs. I should handle these tables the same way I did with Tables 9-14 by wrapping them in code blocks, either by using pdftotext -layout mode or by identifying and isolating the messy table regions directly.

I'm realizing that wrapping tables in layout-mode preformatted blocks makes them much more readable. My approach is to identify each table region in the source by finding the "Table N |" headers, then extract just those page ranges using pdftotext with layout mode to preserve the formatting. I'll combine this with the raw reflowed text I'm already generating to get the best of both. I'm mapping out the table and figure locations across the document—Tables 1 through 8 are scattered through the main sections, with 9-14 in the appendix. For figures, I just need to keep the captions and drop the fragmented chart data that follows. My best approach is to switch into a skip mode whenever I hit a Figure or Table caption, then resume normal processing once I reach actual prose again. This avoids the post-processing complexity and handles the cleanup in a single pass. I'm refining the paragraph processor to be stateful—when it encounters a figure or table caption, it enters a mode where it skips noise paragraphs until hitting real prose again. For tables specifically, I'll insert a layout-mode code block right after the caption, then drop the fragmented content that follows. To make this work, I need to precompute page numbers for each table by scanning the layout text, though I'm also considering a simpler approach: just drop the table fragments entirely and rely on captions, or preserve the most important tables via layout fallback in an appendix-like section at the end.

Actually, I'm realizing that putting tables at the end breaks reading flow—the reader sees "as shown in Table 6" but has to flip to find it. Better to insert the layout-extracted table inline right after the caption for referenced tables, while dropping figure noise entirely. So my plan is to pre-scan the PDF layout to find each table location and capture its full content, then in the main processor, when I encounter a table caption, I'll replace the following noise with the extracted table in a code block. Now I need to check what the actual table formatting looks like in the layout text.

I'm reconsidering the approach—instead of trying to preserve layout mode across the whole body, I'll extract it for sections 1-6 as a single source, then identify tables as indented multi-line blocks and fence them separately. This should handle Tables 1-8 cleanly without the complications of mixed whitespace handling.

The hybrid strategy is cleaner: use raw extraction for prose (which reflows naturally), but pull tables specifically from layout mode where they stay aligned. I can detect tables by their captions like "Table N |..." and map them back to the layout extraction. Then when processing the raw paragraph stream, I'll swap in the fenced table content when I hit a table caption, skip any noise paragraphs after it, and do the same for figures without inserting anything.

For the references section, I need to switch modes once we hit "References"—stop treating heading-like patterns as paragraph breaks since author names will match those patterns. I'll detect the references region by checking if we've encountered a "References" paragraph, then disable the heading-split logic for everything after that point.

Designing the processor...

I need to handle the References section properly by detecting it as a heading, then restrict appendix matching to known titles like "A. Author List and Acknowledgment" and "B. Evaluation Details" so they're recognized as proper headings rather than paragraph content. I'll also drop the standalone "Appendix" line since the next heading already marks the appendix start, and escape any stray # characters that appear in table fragments.

Let me significantly improve the build script with state-tracking and table handling.

I'm noticing that the section headings have disappeared from the output, though the ## Tools heading is still there. The problem seems to be that when I wrap Table 4 in a code block, the layout extraction is picking up lines starting with ## Tools from the schema content—but those should be protected inside the code block and not parsed as markdown. Let me trace through the markdown output to see what's happening around that section.

Readdeepseek-v4.md60 lines

I'm noticing the ## Tools heading is appearing after the closing code fence when it shouldn't be there—it looks like my filtering logic isn't catching this stray paragraph, and it should either be swallowed entirely or wrapped into the table structure above it.

Looking at how the table extraction works in layout mode, I'm capturing Table 4 from its caption all the way until the next figure or table caption appears, but that's pulling in prose that comes after the table ends. The real issue is that I need a better way to detect where a table actually terminates—not just by looking for the next caption, but by spotting when a left-aligned paragraph starts at column 0, which signals the end of the indented table region. Some tables might end at page breaks instead, so I need to handle both cases.

Refining the detection logic: a table ends when I encounter a line at column 0 that isn't a caption keyword and looks like prose—something with enough words and sentence-ending punctuation. I'm updating the extraction function to scan for that first non-indented, prose-like line after the table content, which will properly separate Table 4 from the paragraph about the DeepSeek-V4 series that follows it.

Editedbuild_deepseek_v4.py

Readdeepseek-v4.md40 lines

Editedbuild_deepseek_v4.py

I see there are still a couple of false positives in the heading detection. The issue is that Table 2 contains a numbered example in its "Response Format" column that starts with "2. thinking", and my heading detection is picking this up as a real heading because it matches the pattern of a numbered item. The problem is that the detection logic is running on the split paragraphs from the raw body text, which doesn't distinguish between actual markdown headings and content that just happens to look like one when it appears in a table.

Readdeepseek-v4.md90 lines

I'm noticing that my filtering logic isn't catching all the duplicate table content—the raw paragraph text from Table 2 is still leaking through even though it should be filtered out as noise. There's also a malformed heading appearing from the table's raw data that I need to clean up.

The real issue is that I'm only swallowing noise paragraphs immediately after captions, but the raw extraction contains fragmented table content scattered throughout that I've already emitted in layout mode. I need a better boundary detection—instead of trying to identify where the raw table region ends semantically, I should skip forward by counting the paragraphs that the layout region already covered, or use the line positions to align them properly.

The cleanest approach is to have the table extraction also return the first prose line that appears after each table, then use that as a marker in the raw text to know where to stop skallowing fragments. I'll modify the table extraction to capture the opening 60 characters of the next prose line as a boundary prefix, then in the main processor, after emitting a table, skip all raw paragraphs until I encounter that boundary text.

Defining the data structure and implementing the boundary detection...

The tricky part is handling Unicode characters and ensuring the prefix match works even when raw paragraphs are reflowed differently than the layout version, but the boundary line should still start the same way.

Editedbuild_deepseek_v4.py

I'm noticing an issue with how Table 1's caption is being parsed — the layout extraction is treating wrapped caption lines as prose because they lack indentation, when actually the caption text extends further than expected. Let me check the layout output to see where Table 1 actually ends.

Looking at the other tables, Table 2 correctly ends when Table 3's caption appears, Table 3 and Table 4 have the right prefixes, and Tables 11-13 are picking up header rows from the appendix which I'm handling separately anyway. The real problem is with my parsing order — I'm checking for headings before applying the swallow mode logic, so a heading that appears before a table caption gets emitted even when it should be suppressed.

I need to restructure the loop so that when I'm in table-swallowing mode, I check if the current paragraph matches the prefix for the next table first. If it does, I exit swallow mode and process it normally. If it doesn't match, I skip it entirely. Only after handling the swallow check should I process headings and captions. This way, content trapped between tables won't accidentally get emitted as headings.

Restructuring the parsing loop...

Editedbuild_deepseek_v4.py

I'm noticing a problem with the file structure — there are only 32 headings visible, but the rest of the document appears to be missing sections like 4.3.2, 5, and 5.1. It looks like the swallow_table mode for Table 1 might have consumed everything after it, since the table's prefix text doesn't appear in the raw file, causing the swallow function to continue all the way to the end and consume all subsequent headings.

Let me look at what Table 1's actual caption says...

I'm looking at the table caption which describes a comparison across three DeepSeek model variants, noting that scores within 0.3 points are treated as equivalent, with formatting conventions for the highest and second-highest values in each row.

The layout extraction is breaking the caption across multiple lines, and my prose detection is incorrectly terminating the table early because the wrapped caption lines meet the criteria for prose (unindented, multiple words). I need to distinguish between caption text that continues across lines and actual table data. The key insight is that captions end at a blank line, so I should skip past the entire caption block before checking whether we've reached the end of the table.

Refining the approach...

I'm refining the parsing logic for table detection. The key insight is that a table block starts with a multi-line caption at column 0, followed by a blank line, then indented table data. The block ends when we hit the first column-0 prose line after a blank line. So I need to walk through the text, track whether we've seen at least one blank line, and only then check for the end condition.

Implementing the table termination logic...

Editedbuild_deepseek_v4.py