split by rmd puts some footnotes outside "page-inner" div

[jtbayly]

I just changed a book to split by rmd instead of section, putting each section into its own file. The output for the bookdown::gitbook version takes each chapter's final section's footnotes and puts them outside the "page-inner" div, thus messing up the footnote styling for that section only. I've tried it on the default and dev versions of bookdown, as well as the packaged version of Pandoc and Pandoc 2.10.1. It's the same all around.

Here's a simple demonstration book with the problem visible in sections 1.3 and 2.2, but all other sections working fine:
bd-footnote-problem.zip

If you switch back to split by section and rebuild, the problem disappears.

xfun::session_info('bookdown')

R version 4.0.2 (2020-06-22)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Mojave 10.14.6

Locale: en_US.UTF-8 / en_US.UTF-8 / en_US.UTF-8 / C / en_US.UTF-8 / en_US.UTF-8

Package version:
  base64enc_0.1.3 bookdown_0.20.6 digest_0.6.25   evaluate_0.14   glue_1.4.2     
  graphics_4.0.2  grDevices_4.0.2 highr_0.8       htmltools_0.5.0 jsonlite_1.7.1 
  knitr_1.29      magrittr_1.5    markdown_1.1    methods_4.0.2   mime_0.9       
  rlang_0.4.7     rmarkdown_2.3.5 stats_4.0.2     stringi_1.5.3   stringr_1.4.0  
  tinytex_0.25    tools_4.0.2     utils_4.0.2     xfun_0.17       yaml_2.2.1     

---

By filing an issue to this repo, I promise that

• I have fully read the issue guide at https://yihui.org/issue/.
• I have provided the necessary information about my issue.
  • If I'm asking a question, I have already asked it on Stack Overflow or RStudio Community, waited for at least 24 hours, and included a link to my question there.
  • If I'm filing a bug report, I have included a minimal, self-contained, and reproducible example, and have also included xfun::session_info('bookdown'). I have upgraded all my packages to their latest versions (e.g., R, RStudio, and R packages), and also tried the development version: remotes::install_github('rstudio/bookdown').
  • If I have posted the same issue elsewhere, I have also mentioned it in this issue.
• I have learned the Github Markdown syntax, and formatted my issue correctly.

I understand that my issue may be closed if I don't fulfill my promises.

[jtbayly]

This appears to be closely related to #662 (Formatting of the last element before a new section gets distorted in a GitBook). Perhaps even a duplicate, although in that case he doesn't it only being a problem with split by: rmd.

[cderv]

Yes I can confirm that there is something not intended in the splitting process and the relocating of footnotes. I took some time to look on what is going on, and see if it is easy - but it is not.

I think this is because there is the level 1 <div> which opens in the first Rmd file 01-

<div id="chapter-1" class="section level1" number="1"> 

but your other files does not contains level 1 heading so the closing </div> will be in the third file 03-, and that is causing trouble with the relocation of footnotes, because this closing div will not be associated the level 1 it was before splitting but will close a <section> causing the issue here.

I am not really sure bookdown support writting Rmd files without first level header - There is a warning about that was introduced, I think to warn against this sort of non supported behavior - see #305

So I am not really sure it is a bug - Each Rmd files must contains a level 1 header and split_by: rmd can be used if you want to use the rmd file name for your html file name.

[jtbayly]

Thanks for taking a look. I'm bummed it's not simple to change the behavior.

I was under the impression that by merging first, that I wouldn't need every file to start with a first level header. Indeed, if every file has to start with a first level header, isn't it the case that split by rmd is exactly the same functionality as split by chapter?

[cderv]

Indeed, if every file has to start with a first level header, isn't it the case that split by rmd is exactly the same functionality as split by chapter?

The split is the same but the difference is in the html file names. Per the doc:

split_by
How to name the HTML output files from the book: rmd uses the base filenames of the input Rmd files to create the HTML filenames, e.g. generate ‘chapter1.html’ for ‘chapter1.Rmd’; none means do not split the HTML file (the book will be a single HTML file); chapter means split the file by the first-level headers; section means the second-level headers. For chapter and section, the HTML filenames will be determined by the header ID's, e.g. the filename for the first chapter with a chapter title # Introduction will be ‘introduction.html’; for chapter+number and section+number, the chapter/section numbers will be prepended to the HTML filenames, e.g. ‘1-introduction.html’ and ‘2-1-literature.html’.

By curiosity, why the need of multiple Rmd files for one chapter ?

[jtbayly]

That's odd. "Split by" doesn't seem to imply that it is about naming, but rather where the files are broken up into new web pages. That's what I took away from the bookdown book saying "The main difference between rendering in R Markdown and bookdown is that a book will generate multiple HTML pages by default. To change the way the HTML pages are split, the split_by argument can be specified. This defaults to split_by: chapter, but readers may prefer to use split_by: section if there are many sections within chapters, in which case a chapter page may be too long."

Nothing about naming is mentioned, and in my case, I don't care what the html files are named. I just want the experience to be good for the reader—not empty pages, and not super long pages. Currently I appear to be stuck having one or the other of those:

I have a book where I need to go back and forth from splitting by chapter or splitting by section. Because the content is like this:

# Chapter 1

Long content that should be split from the following sections so it doesn't get too long on one webpage.

## Chapter 1 Section 1

Plenty of content should be split from next section.

## Chatper 1 Section 2

More long content. Split after this, too.

#Chapter 2
(no content here, so no reason to split)
## Chapter 2 Section 1

Long content. Should split after this.

## Chapter 2 Section 2

More long content. Split after this.

# Chapter 3
(Again, no content here, and no reason to split)
## Chapter 3 Section 1

Additional content and sections...

Splitting by chapter doesn't work, because they are way too long. Splitting by section doesn't work, because then I end up with very confusing pages that just say "Chapter 2" or "Chapter 3" with no other content. Those pages should also contain the content of the first section in the chapter.

Split by rmd does exactly what I want. Except for this footnote problem.

By the way, I'm not the only person using it this way:
https://stackoverflow.com/questions/46814182/bookdown-is-it-possible-to-split-by-at-the-subsection-level
https://stackoverflow.com/questions/47064398/r-bookdown-one-chapter-per-markdown-file-rmd
Also, see this issue, (where I gave incorrect info. Oops. I better correct that.): #619 Use rmarkdown files as second-level heading

[jtbayly]

By the way, if you have other suggestions for how I could handle this, I'm all ears.

[cderv]

Thanks for the explanation and the links toward other use case. I see how this can be useful.

Split by rmd does exactly what I want. Except for this footnote problem.

There could be other issue you are not seeing because of the splitting. Basically, bookdown is splitting some html file into several parts and having level 1 header only in some file but not all creates some undesired html structure.

@yihui when implementing split_by, did you try to make it possible to split by Rmd file as desired here ? I see this as challenging but maybe it is not. Do you think it is something we should look into again ?

By the way, if you have other suggestions for how I could handle this, I'm all ears.

I am curious if you tried using (PART) special header (https://bookdown.org/yihui/bookdown/markdown-extensions-by-bookdown.html#special-headers) This would allow to make all you level up to one level, and having some of your chapter 2 become chapter one, in their own Rmd.

[jtbayly]

There could be other issue you are not seeing

You're right. Upon closer examination, it does appear there is a bit of trouble in the HTML. But very little that I saw. Mostly it looks like some elements didn't get named properly.

I am curious if you tried using (PART) special header

I haven't. I'll give it a try.

[jtbayly]

PART does not work for a few reasons.

1. We need to support PDF, HTML, and ePub, but PART doesn't work in ePub. Even if I was willing to give that up...
2. PART doesn't allow putting content inside of it without an h1 or h2. So my example from above fails at the very beginning when I attempt to use PART:

# PART 1

Long content that should be split from the following sections so it doesn't get too long on one webpage.

I would need to make up a meaningless "chapter" before I could put the content in. This might not seem like a high hurdle, but my use of bookdown is almost exclusively to put high-quality reproductions of old books available online. I don't have the freedom to just change the book by making up and adding a chapter title.

Perhaps the title of this issue should be changed to "Feature request: allow Rmd files to start with h2"?

Would a pull request be welcome if I can find somebody to work on that feature?

[Pindar777]

@cderv I'm also using multiple rmd files for one chapter. As a matter of fact the #Chapter headline often stands for its own. The reason is that I can delegate the creation and maintenance of content of a long chapter to different people. This approach works really nice with bs4_book. There is a clean structure on the left side and a detailed toc on the right one.

@jtbayly Thanks to this issue I now have an idea why sometimes the footnotes are beneath the navigation arrows in a bs4_book.