Documentation Production using Open Source Tools Elena Zannoni, - - PowerPoint PPT Presentation

▶
documentation production using open source tools
SMART_READER_LITE
LIVE PREVIEW

Documentation Production using Open Source Tools Elena Zannoni, - - PowerPoint PPT Presentation

Aug 03, 2023 193 likes •655 views

<Insert Picture Here> Documentation Production using Open Source Tools Elena Zannoni, elena.zannoni@oracle.com Stefan Hinz, stefan.hinz@oracle.com Paul Du Bois, paul.dubois@oracle.com CONFIDENTIAL ORACLE HIGHLY RESTRICTED 1

slide-1
SLIDE 1

<Insert Picture Here>

CONFIDENTIAL – ORACLE HIGHLY RESTRICTED 1

Documentation Production using Open Source Tools

Elena Zannoni, elena.zannoni@oracle.com Stefan Hinz, stefan.hinz@oracle.com Paul Du Bois, paul.dubois@oracle.com

slide-2
SLIDE 2

LinuxCon Japan 2015

Introduction

  • Documentation processes for Oracle Linux, MySQL, Oracle VM,

and other virtualization products

  • Other Oracle documentation teams use different technology
  • All our documentation is public on the Oracle Technology

Network (OTN)

  • MySQL documentation is also on dev.mysql.com
  • Formats published: XHTML, WebHelp, PDF, ePUB, man pages
  • Toolchain in use for many years (with many improvements along

the way)

slide-3
SLIDE 3

3 LinuxCon Japan 2015

Infrastructure Overview

  • SVN (Subversion) for all documentation and all tools
  • Store product documentation sources, images
  • Plus, common repository for all our tools, called mysqldoc-toolset
  • Build scripts also stored, mysqldoc-tools
  • Planning to move to GIT
  • DocBook XML as the base documentation format
  • Currently DocBook XML 4.5
  • DocBook XSL style sheets for processing
  • Currently DocBook XSL 1.78.1
  • Customized XSL style sheets for output customization
  • To meet Oracle publication style requirements, accessibility, legal

notices, etc. Using XSL, Perl, and Python

  • Set of Makefiles and make targets for building
  • Fully automated build & publishing system written in Ruby
slide-4
SLIDE 4

4 LinuxCon Japan 2015

Writer's Workflow: Overview

  • Checkout documentation repositories
  • One repository per product
  • Some common repositories needed for stylesheets and for local

building of the output formats

  • Edit XML files
  • Validate resulting XML files
  • Locally build at least one output format
  • Check modified XML files into repository
  • Monitor the automated build to make sure all is well
  • Retrieve output and publish externally
slide-5
SLIDE 5

5 LinuxCon Japan 2015

Writer's Workflow: Authoring

  • DocBook XML is a markup language
  • We use DocBook XML DTD 4.5
  • For editing: use emacs, VI or Oxygen XML Editor
  • Most of our writers prefer Oxygen because of its builtin features such

as:

  • Completion of XML tags
  • Integrated SVN client
  • Integrated XML processing (previews for different formats)
  • Multi platform (runs on MacOS, Linux, Windows..)
  • Inexpensive
slide-6
SLIDE 6

6 LinuxCon Japan 2015

XML File in Oxygen: English

slide-7
SLIDE 7

7 LinuxCon Japan 2015

XML File in Oxygen: Japanese

slide-8
SLIDE 8

8 LinuxCon Japan 2015

Writer's Workflow: Validation & Local Testing

  • This sequence of operations is done locally on author's machine
  • Author validates the XML file –

making sure the XML file is valid DocBook XML 4.5

  • This is done using a program called xmllint
  • Invoked via “make <filename>.valid”
  • Author builds one or more output formats locally (for instance)
  • “make <filename>.html”
  • “make <filename>.pdf”
slide-9
SLIDE 9

9 LinuxCon Japan 2015

Overview of Make Targets

  • Several Makefile targets to facilitate the workflow
  • Make <filename>.valid – verify xml is valid
  • Make <filename>.prepped – preprocessing step (include files, xref..)
  • Make <filename>.format – format file to standard xml (indentation,...)
  • Make <filename>.html
  • Make <filename>.xhtml
  • Make <filename>.webhelp
  • Make <filename>.pdf
  • Make depend: generate list of doc dependencies
  • Make collected: build xhtml file with ID references resolved to URLs
  • Make clean
  • […] plus many flavors of each format
slide-10
SLIDE 10

10 LinuxCon Japan 2015

Resolving References

We call this step ID mapping

  • References to sections in same document or in other documents

must be resolved to correct URLs before building the output targets

  • References look like:
  • <xref linkend="option-file-options"/> (section ID)
  • <xref linkend="refman:replication"/> (file name and section ID)
  • Build the “prepped” file: intermediate XML file with the references

resolved to correct URLs.

  • Build the output formats from the prepped file.
  • Topic for another talk....it involves building more intermediate files

and dependencies

slide-11
SLIDE 11

11 LinuxCon Japan 2015

Building Output Formats

  • Process book.xml (which is a DocBook XML 4.5 file) to an output

format

  • This uses a program called xsltproc
  • That program uses the DocBook XSL Style Sheets

to find out how to do the transformation to another format

  • We use the DocBook XSL Style Sheets 1.78.1
slide-12
SLIDE 12

LinuxCon Japan 2015

Basic Output Formatting Workflow

XSLTPROC XHTML XHTML/ webhelp ePub man PDF .fo FOP XML your-html.xsl your-webhelp.xsl your-epub.xsl your-man.xsl your-fo.xsl Lucene webhelp jQuery

slide-13
SLIDE 13

13 LinuxCon Japan 2015

Writer's Workflow: Final Steps

  • If all looks good, format the XML file according to standards:
  • “make <filename>.format”
  • Check the modified XML file into the repository
  • This action will trigger an official build in the central build system
  • Check for build exit status
  • Publish the resulting books if build successful
slide-14
SLIDE 14

14 LinuxCon Japan 2015

Build System

  • Once the file has been checked in to the central SVN repository

various output formats are built from it – this is fully automated

  • Output formats include HTML, XHTML, WebHelp, EPUB, and PDF
  • The build system is written in Ruby. More precisely dRuby with Rinda.
  • For some things that are hard to do in XSL, it uses Perl
  • For the transformation of huge XML documents to HTML, it uses

Python

  • To build the PDF output format, it uses FOP – we use FOP 1.1
slide-15
SLIDE 15

15 LinuxCon Japan 2015

Build System Architecture Highlights

Central Doc repo

Build Master Build Slave

Local Copy

  • f Doc repo

Commit Notification Checkout

Commit Monitor

Targets to Build Built Targets (Status & Output) Revision # changed

Build Control

Repo to Update Async Commands User/Web UI Writer Commit

slide-16
SLIDE 16

16 LinuxCon Japan 2015

Build System in a Nutshell

  • Communication between master, slave, commit monitor

happens via messages

  • Messages are tuples
  • Commit monitor notices source repo changes and informs

build master

  • Build master dispatches a request for repo update to a

slave

  • Build master dispatches a build to a slave
  • Slave does the build and returns built files and status to

master

slide-17
SLIDE 17

17 LinuxCon Japan 2015

Implementation: Rinda Tuple Space

  • Communication server operates on the TupleSpace (in

Rinda)

  • Every client interacts with the TupleSpace via the server.
  • Clients in the build system: Build master, Build slaves, Build

Control, Commit Monitor

  • Communication done by sending and receiving tuples

to/from tuplespace

  • Tuples have destination/sender plus message body
slide-18
SLIDE 18

18 LinuxCon Japan 2015

Configuration Files

  • Build slaves conf file: list of repositories to checkout/update
  • Targets conf file: list of build targets
  • Commit monitor conf file: list of repositories to monitor for

commits

slide-19
SLIDE 19

19 LinuxCon Japan 2015

Commit-monitor and Slave Conf

Fragment from commit-monitor configuration file: [...] "repositories" => { "linux" => { "check-type" => "svn info", "repo-url" => "#{repo_url_prefix}/linux/trunk" }, [...] } Fragment from build slaves configuration file: "repositories" => { "linux" => { "basedir" => "#{repo_parent}/linux", "update-cmd" => update_cmd, "documents" => { "uek3-qu4-relnotes" => "uek3/qu4", "uek3-qu5-relnotes" => "uek3/qu5", […] }}}

Command to check revision numbers Command to get local copy of repo List of all possible documents to be built Path to the repo

slide-20
SLIDE 20

20 LinuxCon Japan 2015

Target Configuration Example

"linux" => { "lang-code" => "en", "documents" => { "uek3-qu5-relnotes" => { "output-directory" => "linux/uek3-qu5-relnotes", "target-base" => "relnotes", "document-barriers" => std_document_barriers, "goal-targets" => [ [ "%T.html-embed", 900 ], [ "%T.webhelp", 900 ], [ "%T.xhtml-dir", 900 ], [ "%T.xhtml-otn", 900 ], [ "%T.xhtml-review", 900 ], [ "%T.pdf", 910 ], [ "%T-review.pdf", 910 ], [ "%T.epub", 920 ], [ "%T.txt", 920 ], ], } } }

Multiple Languages Possible

Output directory in the output tree <tree-root>/<output-directory>/<lang-code>/ Standard prerequisites used List of targets “make <target-name>” will be used to build Each target has a priority Lower number == higher priority

slide-21
SLIDE 21

21 LinuxCon Japan 2015

Build Targets

For each document there is a series of targets to be built

  • Goal Targets: targets that correspond to desired output

formats (HTML, XHTML, PDF, etc). Output is saved and returned.

  • Document Barriers: targets that must be built successfully

before goal targets can be built. For instance making sure that the source document (XML file) is valid. No output is produced.

  • docs_valid
slide-22
SLIDE 22

22 LinuxCon Japan 2015

Communication: Master - Slave

  • Master: assign_document_barrier to slave
  • Master: assign_goal_target to slave
  • Slave: accept document_barrier from master
  • Slave: accept goal_target from master
  • Slave: return terminated_target to master
  • Master: accepts terminated_target from slave
  • Tuples
  • Document barrier: [repo_name, doc_name, target_name, build_id]
  • Goal target: [repo_name, doc_name, target_name, build_id, priority,

signature]

slide-23
SLIDE 23

23 LinuxCon Japan 2015

Tuples Examples

  • Registry Tuples
  • [:registry, 118819019, "2015-05-27 13:14:25", "slave@oter02/2349",

"slave", 1500]

  • [:registry, 118819883, "2015-05-27 13:17:09", "master@oter02/27074",

"master", 240]

  • Message Tuples
  • [:message, 118820762, "2015-05-27 13:17:17", "master@oter02/27074",

"slave@oter02/2349", "build-document-barrier", ["linux", "ol6-admin", "admin-collected.xml", 3542031]]

  • [:message, 118822774, "2015-05-27 13:17:30", "slave@oter02/2349",

"master@oter02/27074", "target-build-result", ["linux", "ol6-admin", "admin.xhtml-dir", 3542101, 3, nil, 0, "admin.xhtml-dir is up to date ...

  • [:message, 118822851, "2015-05-27 13:17:31", "master@oter02/27074",

"slave@oter02/2349", "build-goal-target", ["linux", "uek2-qu4-relnotes", "relnotes.webhelp", 3542117, 900, "9c68cc4d5077069b62c04...

slide-24
SLIDE 24

24 LinuxCon Japan 2015

Commit Monitor & Repo Update Example

113385876 2015-05-18 08:32:36: master@oter02/27074: received message: commit from commit-monitor@oter02/4108 113385877 2015-05-18 08:32:36: master@oter02/27074: a commit occurred for repository linux 113385878 2015-05-18 08:32:36: master@oter02/27074: signal slaves to update repository linux 113385879 2015-05-18 08:32:36: master@oter02/27074: tell slave@oter02/25453 to update repository linux 113385881 2015-05-18 08:32:36: master@oter02/27074: slave@oter02/25453: number of targets reclaimed: 0 113385882 2015-05-18 08:32:36: master@oter02/27074: slave@oter02/25453: load capacity available = 0 113385883 2015-05-18 08:32:36: master@oter02/27074: write target-status report 113385978 2015-05-18 08:32:38: slave@oter02/25453: received update notification for repository linux 113385989 2015-05-18 08:32:38: slave@oter02/25453: schedule update for repository linux, caller = ["/usr/local/bin/builds.rb:836:in `block in main_loop'", "/usr/local/bin/builds.rb:676:in `loop'", "/usr/local/bin/builds.rb:676:in `main_loop'", "/usr/local/bin/builds.rb:978:in `<main>'"] 113386024 2015-05-18 08:32:38: slave@oter02/25453: number of pending updates: 1 113386042 2015-05-18 08:32:38: slave@oter02/25453: repositories to update: ["linux"] 113386119 2015-05-18 08:32:39: slave@oter02/25453: MutexArray: value "linux" dequeued, size now 0 113386120 2015-05-18 08:32:39: slave@oter02/25453: START repository update thread 113386121 2015-05-18 08:32:39: slave@oter02/25453: perform update for repository linux, caller = ["/usr/local/bin/builds.rb:861:in `block (2 levels) in main_loop'"] 113386122 2015-05-18 08:32:39: slave@oter02/25453: END repository update thread 113386228 2015-05-18 08:32:41: slave@oter02/25453: update succeeded for repository linux 113387039 2015-05-18 08:32:59: master@oter02/27074: received message: repository-update-result from slave@oter02/25453 113387040 2015-05-18 08:32:59: master@oter02/27074: slave slave@oter02/25453 has updated repository: linux, status = 0 113387041 2015-05-18 08:32:59: master@oter02/27074: write slave-status report 113387042 2015-05-18 08:32:59: master@oter02/27074: done writing slave-status report 113387043 2015-05-18 08:32:59: master@oter02/27074: write target-status report 113387044 2015-05-18 08:33:00: master@oter02/27074: done writing target-status report

slide-25
SLIDE 25

25 LinuxCon Japan 2015

Document Barrier Building

113388622 2015-05-18 08:33:17: master@oter02/27074: ASSIGN document barrier: slave@oter02/25453:linux/uek3-qu5- relnotes/relnotes.valid (id = 3381088) 113388683 2015-05-18 08:33:18: slave@oter02/25453: received message: build-document-barrier from master@oter02/27074 113388684 2015-05-18 08:33:18: slave@oter02/25453: message data: ["linux", "uek3-qu5-relnotes", "relnotes.valid", 3381088] 113388685 2015-05-18 08:33:18: slave@oter02/25453: ACCEPT document barrier linux/uek3-qu5-relnotes/relnotes.valid (id = 3381088) 113388790 2015-05-18 08:33:20: slave@oter02/25453: linux/uek3-qu5-relnotes: START DOCUMENT BARRIER: make -j1 relnotes.valid 113388803 2015-05-18 08:33:20: slave@oter02/25453: linux/uek3-qu5-relnotes: END DOCUMENT BARRIER: relnotes.valid (status = 0) 113388805 2015-05-18 08:33:20: slave@oter02/25453: linux/uek3-qu5-relnotes: TERMINATED DB: relnotes.valid (status = 0) 113388806 2015-05-18 08:33:20: slave@oter02/25453: target has terminated: linux/uek3-qu5-relnotes/relnotes.valid 113388807 2015-05-18 08:33:20: slave@oter02/25453: write result tuple to master: linux/uek3-qu5-relnotes/relnotes.valid 113388811 2015-05-18 08:33:20: slave@oter02/25453: done writing result tuple to master: linux/uek3-qu5-relnotes/relnotes.valid 113389898 2015-05-18 08:33:39: master@oter02/27074: received message: target-build-result from slave@oter02/25453 113389899 2015-05-18 08:33:39: master@oter02/27074: TERMINATED: slave@oter02/25453:linux/uek3-qu5- relnotes/relnotes.valid (id = 3381088) (result = TARGET_UP_TO_DATE) 113389900 2015-05-18 08:33:39: master@oter02/27074: accept terminated target: 1/slave@oter02/25453:linux/uek3-qu5- relnotes/relnotes.valid

slide-26
SLIDE 26

26 LinuxCon Japan 2015

Build Goal Target: .pdf

114469865 2015-05-19 12:54:10: master@oter02/27074: ASSIGN goal target: slave@oter02/26410:linux/uek3-qu5-relnotes/relnotes.pdf (id = 3414551) 114470447 2015-05-19 12:54:40: slave@oter02/26410: message data: ["linux", "uek3-qu5-relnotes", "relnotes.pdf", 3414551, 910, "0bbbc97586c3b26ee2b2d79aaf7d4817"] 114470448 2015-05-19 12:54:40: slave@oter02/26410: ACCEPT goal target linux/uek3-qu5-relnotes/relnotes.pdf (id = 3414551) 114470449 2015-05-19 12:54:40: slave@oter02/26410: received message: build-goal-target from master@oter02/27074 114470828 2015-05-19 12:54:44: slave@oter02/26410: linux/uek3-qu5-relnotes: ADD DEPENDENCIES for goal target relnotes.pdf 114470861 2015-05-19 12:54:44: slave@oter02/26410: linux/uek3-qu5-relnotes: Execute: makedeps.rb relnotes.pdf 114470913 2015-05-19 12:54:45: slave@oter02/26410: linux/uek3-qu5-relnotes: INITIAL DEPENDENCIES for relnotes.pdf: ["relnotes.pdf", "relnotes.fo"] 114470990 2015-05-19 12:54:46: slave@oter02/26410: linux/uek3-qu5-relnotes: UNBUILT DEPENDENCIES for relnotes.pdf: ["relnotes.pdf", "relnotes.fo"] 114471041 2015-05-19 12:54:47: slave@oter02/26410: linux/uek3-qu5-relnotes: add dependency list for relnotes.pdf: ["relnotes.pdf", "relnotes.fo"] 114471073 2015-05-19 12:54:47: slave@oter02/26410: linux/uek3-qu5-relnotes: add dependency list for relnotes.fo: ["relnotes.fo"] 114471142 2015-05-19 12:54:48: slave@oter02/26410: linux/uek3-qu5-relnotes: job PIDs: [13498] 114471278 2015-05-19 12:54:51: slave@oter02/26410: linux/uek3-qu5-relnotes: build relnotes.pdf: dependencies: ["relnotes.pdf", "relnotes.fo"] 114471315 2015-05-19 12:54:52: slave@oter02/26410: linux/uek3-qu5-relnotes: START GOAL TARGET (dependency): make -j1 relnotes.fo 114471340 2015-05-19 12:54:53: slave@oter02/26410: linux/uek3-qu5-relnotes: job PIDs: [14833] 114471386 2015-05-19 12:54:54: slave@oter02/26410: linux/uek3-qu5-relnotes: END GOAL TARGET (dependency): relnotes.fo (status = 0) 114471387 2015-05-19 12:54:54: slave@oter02/26410: linux/uek3-qu5-relnotes: TERMINATED: relnotes.fo (status = 0) 114471400 2015-05-19 12:54:56: slave@oter02/26410: linux/uek3-qu5-relnotes: job PIDs: [15321] 114471458 2015-05-19 12:54:58: slave@oter02/26410: linux/uek3-qu5-relnotes: build relnotes.pdf: dependencies: ["relnotes.pdf"] 114471471 2015-05-19 12:54:58: slave@oter02/26410: linux/uek3-qu5-relnotes: START GOAL TARGET: make -j1 relnotes.pdf 114471493 2015-05-19 12:55:00: slave@oter02/26410: linux/uek3-qu5-relnotes: job PIDs: [15321, 15581] 114471838 2015-05-19 12:55:17: slave@oter02/26410: linux/uek3-qu5-relnotes: END GOAL TARGET: relnotes.pdf (status = 0) 114471874 2015-05-19 12:55:18: slave@oter02/26410: linux/uek3-qu5-relnotes: TERMINATED: relnotes.pdf (status = 0)

slide-27
SLIDE 27

27 LinuxCon Japan 2015

Complete Build and Store Results

114472036 2015-05-19 12:55:23: slave@oter02/26410: linux/uek3-qu5-relnotes: old, new signatures (relnotes.pdf) = 0bbbc97586c3b26ee2b2d79aaf7d4817, 983d383f7ac54121602070a546aca16e 114472059 2015-05-19 12:55:23: slave@oter02/26410: linux/uek3-qu5-relnotes: content has changed, package and return: relnotes.pdf 114472086 2015-05-19 12:55:24: slave@oter02/26410: target has terminated: linux/uek3-qu5-relnotes/relnotes.pdf 114472103 2015-05-19 12:55:25: slave@oter02/26410: write result tuple to master: linux/uek3-qu5-relnotes/relnotes.pdf 114472169 2015-05-19 12:55:27: slave@oter02/26410: done writing result tuple to master: linux/uek3-qu5-relnotes/relnotes.pdf 114473189 2015-05-19 12:55:52: master@oter02/27074: TERMINATED: slave@oter02/26410:linux/uek3-qu5-relnotes/relnotes.pdf (id = 3414551) (result = TARGET_UP_TO_DATE) 114473190 2015-05-19 12:55:52: master@oter02/27074: accept terminated target: 2/slave@oter02/26410:linux/uek3-qu5- relnotes/relnotes.pdf 114473191 2015-05-19 12:55:52: master@oter02/27074: old/new signatures (linux/uek3-qu5-relnotes/relnotes.pdf) = 0bbbc97586c3b26ee2b2d79aaf7d4817/983d383f7ac54121602070a546aca16e 114473192 2015-05-19 12:55:52: master@oter02/27074: content changed, unpack and store: linux/uek3-qu5-relnotes/relnotes.pdf 114473193 2015-05-19 12:55:52: master@oter02/27074: uek3-qu5-relnotes/en, target relnotes.pdf: CONTENT size: 76352 114473194 2015-05-19 12:55:52: master@oter02/27074: status reported: up to date/linux/uek3-qu5-relnotes/relnotes.pdf

slide-28
SLIDE 28

28 LinuxCon Japan 2015

Repository / Slave Status

REPO_OFFLINE REPO_UPDATING REPO_ONLINE After commit repo out of date Update Failed Update Succeeded Initialize Build new target

slide-29
SLIDE 29

29 LinuxCon Japan 2015

Target Status

T_OUT_OF_DATE T_FAILED T_UNBUILDABLE T_UP_TO_DATE T_BUILDING Start Slave can build target and REPO_ONLINE Build OK Build Failed (*) Dequarantined All REPO_OFFLINE or REPO_UPDATING No repo on slaves No rule to build target Failed barrier REPO_ONLINE (*) Many different reasons recorded in the status of the target

slide-30
SLIDE 30

30 LinuxCon Japan 2015

Web Interface – Global Slave Status

slide-31
SLIDE 31

31 LinuxCon Japan 2015

Web Interface – Slave Status

slide-32
SLIDE 32

32 LinuxCon Japan 2015

Web Interface – Target Status

slide-33
SLIDE 33

33 LinuxCon Japan 2015

Web Interface - Commits Report

slide-34
SLIDE 34

34 LinuxCon Japan 2015

Quarantining

  • After target is built it is checked for significant differences in size from

the previous version.

  • If the target is a single file, the sizes are compared
  • If the target is multiple files (like multi chapter HTML) the number of

files and the total size are compared

  • If size difference is above a certain threshold (currently 10% of size, but

could vary) then the built target is not stored in output tree

  • Stored in quarantine tree instead
  • Must be manually inspected and dequarantined
  • If dequarantined, it is moved to the output tree (and target status

changes to T_UP_TO_DATE)

slide-35
SLIDE 35

35 LinuxCon Japan 2015

Quarantine Message Sequence

113321371 2015-05-17 22:39:00: master@oter02/27074: old/new signatures (mysql-relnotes/connector-c/relnotes.a4.pdf) = 06ff395d784c6 5bfc4fb1c52056c6be7/46de15c9c0e4c4a4ab4cdf5dba0c0733 113321372 2015-05-17 22:39:00: master@oter02/27074: content changed, unpack and store: mysql-relnotes/connector- c/relnotes.a4.pdf 113321373 2015-05-17 22:39:00: master@oter02/27074: connector-c/en, target relnotes.a4.pdf: CONTENT size: 36486 113321379 2015-05-17 22:39:00: master@oter02/27074: WARNING: output is suspicious; copy to quarantine tree 113321387 2015-05-17 22:39:00: master@oter02/27074: WARNING: quarantining connector-c/en, target relnotes.a4.pdf:

  • ld, new targets had 12% size difference (old=65533, new=74007)

113321388 2015-05-17 22:39:00: master@oter02/27074: WARNING: relnotes.a4.pdf failed, content was quarantined 113321375 2015-05-17 22:39:00: slave@oter02/25453: done writing result tuple to master: mysql-relnotes/connector- c/relnotes.a4.pdf 113321402 2015-05-17 22:39:00: master@oter02/27074: status reported: failed/mysql-relnotes/connector- c/relnotes.a4.pdf

slide-36
SLIDE 36

36 LinuxCon Japan 2015

Dequarantine Message Sequence

113322916 2015-05-17 22:39:36: buildc@oter02/4598: send message: dequarantine 113322945 2015-05-17 22:39:36: master@oter02/27074: received message: dequarantine from buildc@oter02/4219 113322946 2015-05-17 22:39:36: master@oter02/27074: dequarantine content for mysql-relnotes/connector-c/relnotes.a4.pdf 113322947 2015-05-17 22:39:36: master@oter02/27074: look for content under the relnotes/connector-c/en directory 113322948 2015-05-17 22:39:36: master@oter02/27074: dequarantine command: rsync --archive --delete /data0/users/mysqldoc/quarantine-tree/relnotes/connector-c/en/pdf-a4/ /data0/users/mysqldoc/staging-tree/relnotes/connector- c/en/pdf-a4/ 113322956 2015-05-17 22:39:36: master@oter02/27074: dequarantine command: rsync --archive /data0/users/mysqldoc/quarantine-tree/relnotes/connector-c/en/pdf-a4.tar.gz /data0/users/mysqldoc/quarantine- tree/relnotes/connector-c/en/pdf-a4.zip /data0/users/mysqldoc/staging-tree/relnotes/connector-c/en/pdf-a4/.. 113322964 2015-05-17 22:39:36: master@oter02/27074: dequarantine command: rm -rf /data0/users/mysqldoc/quarantine- tree/relnotes/connector-c/en/pdf-a4 /data0/users/mysqldoc/quarantine-tree/relnotes/connector-c/en/pdf-a4.tar.gz /data0/users/mysqldoc/quarantine-tree/relnotes/connector-c/en/pdf-a4.zip 113322965 2015-05-17 22:39:36: master@oter02/27074: mark mysql-relnotes/connector-c/relnotes.a4.pdf up to date

slide-37
SLIDE 37

37 LinuxCon Japan 2015

Publishing

  • The publication process is two-fold (very Oracle specific):
  • Publication on OTN (Oracle Technology Network) is done manually

by the authors, using a company wide tool

  • Publication on the MySQL Developer Zone (dev.mysql.com/doc) is

automatic; books are published to the MySQL Web team's staging server using cron-driven shell scripts

  • Not going into details here
slide-38
SLIDE 38

38 LinuxCon Japan 2015

What is Published

  • Output formats use “standard” technologies:
  • HTML (with CSS and PNG images)
  • XHTML (with CSS and PNG images)
  • PDF – standalone binary files
  • EPUB – standalone binary files
  • HTML download files (tar.gz and zip) – standalone binary files
  • Man pages for download (tar.gz and zip) – standalone binary files
  • HTML and PDF for download in RPM format – standalone binary

files

  • Info (gz and zip) – standalone binary files
  • WebHelp (see next slide)
slide-39
SLIDE 39

39 LinuxCon Japan 2015

WebHelp Format

  • XHTML with CSS and JavaScript
  • Search queries done by JavaScript in the browser
  • Results are prepared by comparing the query with a search 'index' also in

JavaScript

  • Build process:
  • Uses DocBook XSL Style Sheets 1.78.1 and a Java tool called Lucene 3.0
  • Lucene 3.0 is part of the DocBook XSL 1.78.1 Style Sheets
  • Lucene creates the JavaScript search index used in WebHelp
  • Published output format:
  • XHTML with CSS
  • JavaScript search index (index1.js, index2.js, index3.js) (Lucene's output)
  • jQuery 1.7.2 and 1.8.2 (JavaScript & CSS framework): jQuery creates the

TOC

  • jQuery is part of the DocBook XSL 1.78.1 Style Sheets
slide-40
SLIDE 40

40 LinuxCon Japan 2015

Customized Webhelp Example - English

slide-41
SLIDE 41

41 LinuxCon Japan 2015

Customized Webhelp Example - Japanese

slide-42
SLIDE 42

42 LinuxCon Japan 2015

WebHelp References

  • Example output library for a particular product (Oracle Secure Global

Desktop) with webhelp format: http://docs.oracle.com/cd/E51728_01/index.html

  • Used by other projects as well, like OpenStack, Rackspace, etc
  • Included in Docbook stylesheets
  • Was developed as part of Google Summer of Code in 2012
slide-43
SLIDE 43

LinuxCon Japan 2015

Main Resources

  • XSL stylesheets (eXtensible Stylesheet Language)

www.docbook.org

  • Xmllint is part of libxml2. Checks validity of XML file

http://xmlsoft.org/

  • Xsltproc is part of libxslt. Produces output formats based on the

directives in the given stylesheet (xhtml, epub, fo, etc) http://xmlsoft.org/

  • Fop: Apache fop (formatting objects processor). Produces PDF

files from .fo files https://xmlgraphics.apache.org/fop/

  • Lucene: Library for search and indexing (used in Webhelp)

https://lucene.apache.org/

  • jQuery: javascript library for document manipulation. Used by

Webhelp format to do searches within browser https://jquery.org

slide-44
SLIDE 44

44 LinuxCon Japan 2015

Questions?