340 Commits

Author SHA1 Message Date
hugh-obrien
bfe31098cb send html 2018-07-09 14:37:54 +01:00
hugh-obrien
a1f9305047 hard coded a word doc 2018-07-09 14:02:39 +01:00
hugh-obrien
c66d76af2d make it do pdf 2018-07-09 13:52:18 +01:00
hugh-obrien
b8c82620e5 main file hack 2018-07-09 13:45:57 +01:00
hugh-obrien
3b905353d0 main file fix nonsence 2018-07-09 11:40:40 +01:00
hugh-obrien
af6a402a87 fix array nonsense 2018-07-09 11:37:57 +01:00
hugh-obrien
3c639959f4 hack in the command and image 2018-07-09 11:35:19 +01:00
Alberto Fernández-Capel
039d5e01ec Merge pull request #87 from sharelatex/afc-travis-nvmrc
Make travis read the node version from the .nvmrc file
2018-05-01 09:45:55 +01:00
Alberto Fernández Capel
1d38dd3a92 Make travis read the node version from the .nvmrc file
See https://docs.travis-ci.com/user/languages/javascript-with-nodejs/#Specifying-Node.js-versions-using-.nvmrc
2018-05-01 09:25:37 +01:00
James Allen
12c1dc632a Merge pull request #83 from sharelatex/ja-dockerize-dev
Provide hosts as settings and add npm run start script
2018-01-16 17:08:09 +00:00
James Allen
7a6294081d Allow texlive image user to be configured 2018-01-16 10:46:59 +00:00
Brian Gough
7d8a18c46c Merge pull request #82 from sharelatex/bg-log-core-files-as-error
log an error if core file is found in output
2018-01-04 09:22:44 +00:00
Brian Gough
a0d5e6a54b log an error if core file is found in output 2018-01-03 15:41:31 +00:00
James Allen
f58ef67875 Provide hosts and siblings container as environment settings and add npm run start script 2017-12-29 08:08:19 +00:00
Joe Green
6d42e18088 Add a 1 second delay to the smoke tests (#81)
* Add a 1 second delay to the smoke tests

Fixes a race condition where smoke tests exit before container can be attached to.

See here for more info: https://github.com/overleaf/sharelatex/issues/274

* give the smoke tests additional work to do

* escape slashes
2017-12-05 16:51:59 +00:00
Joe Green
ef0db41dae Merge pull request #80 from sharelatex/jg-smoketest-interval
Increase smoke test interval to 30 seconds
2017-11-29 15:34:49 +00:00
Joe Green
3692570df0 Increase smoke test interval to 30 seconds
The smoke tests can sometimes take ~20 seconds to complete, which causes the http POST to time out. This should solve that problem.
2017-11-29 11:01:51 +00:00
Brian Gough
8255997fad Merge pull request #79 from sharelatex/bg-fix-listen-in-acceptance-tests
exit if mock server fails to start
2017-10-25 09:13:33 +01:00
Brian Gough
360e8220ce exit if mock server fails to start 2017-10-20 15:16:35 +01:00
Joe Green
23f4f2175c Update Jenkinsfile 2017-10-16 14:13:51 +01:00
Joe Green
eb35cab72d only alert on master 2017-10-12 16:54:54 +01:00
Brian Gough
48b2548533 Merge pull request #78 from sharelatex/bg-fix-read-logging
fix read logging
2017-10-02 16:12:12 +01:00
Brian Gough
86cc30d8fa fix typo in log message 2017-10-02 15:45:09 +01:00
Brian Gough
60ad425205 move logging from SafeReader into caller
prevent unnecessary logging when looking at headers of files where
hitting the end of the file is expected.
2017-10-02 15:44:00 +01:00
Brian Gough
d63f339fc4 Merge pull request #77 from sharelatex/bg-fix-tikzexternalize-II
fix tikzexternalize ii
2017-10-02 11:19:06 +01:00
Brian Gough
1da918e13c simplify tikzexternalize checks 2017-09-29 17:00:53 +01:00
Brian Gough
d1aa1d84fb keep tikzexternalize files 2017-09-29 16:02:23 +01:00
Joe Green
88eafdf575 Update Jenkinsfile 2017-09-28 13:46:01 +01:00
Brian Gough
d8858cfadd Merge branch 'bg-lock-compiles' 2017-09-28 13:16:29 +01:00
Joe Green
fd0cbb2c52 use npm cache in CI build 2017-09-28 11:51:41 +01:00
Joe Green
bd5a0ef36f Jg jenkinsfile cleanup (#75)
* Update Jenkinsfile

make sure we don't ship unneeded build files

* Update ExampleDocumentTests.coffee

* use node 6.11.2 in jenkins file
2017-09-28 11:50:33 +01:00
Brian Gough
1388093866 Merge pull request #73 from sharelatex/bg-handle-dot-files-in-resource-list
handle dot files in resource list
2017-09-28 09:59:27 +01:00
Joe Green
c3e3e3d8ac Update Jenkinsfile 2017-09-26 11:44:48 +01:00
Brian Gough
23fec68111 use a separate function for hidden file check 2017-09-26 11:03:20 +01:00
Brian Gough
dbeff9a7b8 exclude hidden files from output
express static server doesn't serve them and rejects with 404
2017-09-26 10:42:59 +01:00
Brian Gough
f11468b595 remove stat test for missing files 2017-09-26 09:48:09 +01:00
Brian Gough
0930b1cd8f only exclude clsi-specific files from output list 2017-09-26 09:47:29 +01:00
Brian Gough
a36ec7f54e fix comment 2017-09-25 16:06:45 +01:00
Brian Gough
eaa99c7274 fix unit tests for use of fs-extra 2017-09-25 15:28:31 +01:00
Brian Gough
b0f879d652 lock compile directory 2017-09-22 16:19:33 +01:00
Brian Gough
8305268848 unit tests for ResourceStateManager 2017-09-15 13:42:57 +01:00
Brian Gough
aa5eeb0903 fallback check for missing files
dot files are not examined by OutputFileFinder, so do an extra check to
make sure those exist

also check for any relative paths in the resources
2017-09-15 13:41:56 +01:00
Brian Gough
2af05030f2 Merge pull request #71 from sharelatex/bg-merge-state-and-resource-list-files
merge state and resource list files
2017-09-11 08:54:30 +01:00
Joe Green
d04f93855b Add jenkinsfile (#72)
* create Jenkinsfile

* allow textlive image to be set with env vars

* log error message in test

* use sandboxed compiles variables

* Add SANDBOXED_COMPILES_HOST_DIR var to test config

* add SIBLING_CONTAINER_USER env var
2017-09-08 14:06:04 +01:00
Brian Gough
a2c97e6f9a rename saveProjectStateHash to saveProjectState 2017-09-08 13:56:40 +01:00
Brian Gough
acab9d45a0 log any missing files 2017-09-07 16:54:09 +01:00
Brian Gough
0fac2655f7 fix whitespace 2017-09-07 13:52:34 +01:00
Brian Gough
c1ca32184f log error if state file is truncacted 2017-09-07 13:52:34 +01:00
Brian Gough
97d7d76e61 combine the resource state and resource list
to prevent them getting out of sync
2017-09-07 13:52:34 +01:00
Shane Kilkelly
d865fda6a9 Merge pull request #70 from sharelatex/sk-node-6
Upgrade to node 6.11
2017-08-31 13:35:27 +01:00
Shane Kilkelly
3d053a2e34 Upgrade to node 6.9 2017-08-29 14:30:43 +01:00
Brian Gough
faa2a325cb added logging 2017-08-29 12:09:31 +01:00
James Allen
b42347ea08 Merge pull request #69 from sharelatex/as-update-docker-runner-config
Update docker-runner-sharelatex config
2017-08-24 15:17:16 +02:00
Alasdair Smith
d5b3101637 Update docker-runner-sharelatex config 2017-08-24 13:34:24 +01:00
Brian Gough
c1d1f93453 Merge pull request #66 from sharelatex/bg-compile-from-redis
Write files incrementally
2017-08-23 15:35:56 +01:00
Brian Gough
fc1782e74c read resource files safely
put a limit on the amount of data read
2017-08-18 11:17:01 +01:00
Brian Gough
6921cf25b8 splice state management into ResourceStateManager 2017-08-18 10:22:17 +01:00
Brian Gough
0b9ddb8efe fix whitespace 2017-08-18 09:41:59 +01:00
Brian Gough
e8064f12a1 finish unit test for incremental update 2017-08-18 09:41:43 +01:00
Brian Gough
e4aad90f33 ResourceWriter unit tests (wip) 2017-08-17 16:59:37 +01:00
Brian Gough
a8aaf58e64 test syncType in RequestParser 2017-08-17 15:57:05 +01:00
Brian Gough
5b5f7b0690 avoid adding draft mode more than once 2017-08-17 15:03:37 +01:00
Brian Gough
2b610030d5 store the resource list in a file 2017-08-17 14:53:35 +01:00
Brian Gough
00ddfdf42b fix unit tests 2017-08-09 15:22:44 +01:00
Brian Gough
c25e96bbc3 add comment about syncType/syncState 2017-08-09 15:22:38 +01:00
Henry Oswald
4eb8c107c9 Merge pull request #68 from sharelatex/ho-mkdir-cache-comiles
use grunt to make compiles and cache dirs
2017-08-09 11:07:36 +01:00
Brian Gough
86fa940c97 clean up the state file if no state passed in 2017-08-08 16:29:57 +01:00
Henry Oswald
7cd81ac3df use grunt to make compiles and cache dirs 2017-08-07 16:21:37 +01:00
Henry Oswald
fdc22c9cd2 Merge pull request #67 from sharelatex/revert-65-add-compiles-folder
Revert "Keep compiles and cache directories"
2017-08-07 15:29:30 +01:00
Henry Oswald
c3fe17d0b6 Revert "Keep compiles and cache directories" 2017-08-07 15:29:18 +01:00
Brian Gough
206adc2d04 fix broken unit tests 2017-08-07 15:00:16 +01:00
Brian Gough
6542ce20b6 fix incremental request 2017-08-07 14:32:28 +01:00
Brian Gough
b4be40d061 restrict syncType values to full/incremental 2017-08-07 10:19:56 +01:00
Brian Gough
11898b897e added files out of sync error object 2017-08-03 15:56:59 +01:00
Brian Gough
74c26120b2 use syncType and syncState for clsi state options 2017-08-03 12:00:32 +01:00
Brian Gough
7e1d3d98e7 write files incrementally 2017-08-02 13:46:10 +01:00
Henry Oswald
d5e0ab5a6f Merge pull request #65 from sharelatex/add-compiles-folder
Keep compiles and cache directories
2017-07-28 11:24:36 +01:00
Hayden Faulds
4c105e7826 keep cache directory 2017-07-27 15:54:20 +01:00
Hayden Faulds
cd5adaff51 keep compiles directory 2017-07-27 14:02:24 +01:00
Henry Oswald
e5081df2a9 Revert "change"
This reverts commit 104ce81ebd.
2017-07-23 22:45:04 +01:00
Henry Oswald
104ce81ebd change 2017-07-23 22:42:07 +01:00
Brian Gough
08fd440df5 Merge pull request #63 from sharelatex/bg-fix-tikzmanager-exception
fix tikzmanager exception
2017-07-20 13:22:58 +01:00
Brian Gough
11cd569ed9 stub out unwanted dependency in unit tests 2017-07-18 11:30:22 +01:00
Brian Gough
472531f617 fix exception for empty content in TikzManager 2017-07-18 11:29:59 +01:00
Brian Gough
ea34a1a89d update acceptance test images for texlive 2017 2017-07-13 13:15:51 +01:00
Brian Gough
2e91f07014 update acceptance tests settings to 2017 image 2017-07-12 16:59:33 +01:00
Shane Kilkelly
6f322583f7 Merge branch 'sk-reduce-kill-project-errors' 2017-06-27 10:03:51 +01:00
Shane Kilkelly
a74f4ac1a6 Send a 404 if the project files have gone away when running synctex.
This is semantically nicer than the 500 response which used to be
produced in these circumstances.
2017-06-23 14:46:40 +01:00
Shane Kilkelly
aa1dd2bf05 Killing an already stopped project is not an error
Log a warning instead and continue.
2017-06-20 09:18:15 +01:00
Shane Kilkelly
8e2584bab4 Mock out logger in tests 2017-06-20 08:25:50 +01:00
Brian Gough
f8530da626 Merge pull request #60 from sharelatex/bg-delete-xdv-files
delete intermediate xdv files from xelatex
2017-06-16 09:13:43 +01:00
Brian Gough
2edc015663 delete intermediate xdv files from xelatex 2017-06-15 15:37:45 +01:00
Brian Gough
f94e9989ec Merge pull request #58 from sharelatex/bg-check-dir-before-synctex
check file exists before running synctex
2017-05-31 10:16:06 +01:00
Brian Gough
c62f8b4854 check directory exists and bail out on error 2017-05-31 10:06:27 +01:00
Brian Gough
2d389130cc Merge pull request #59 from sharelatex/bg-reduce-clsi-error-reporting
don't report compile timeouts to sentry
2017-05-30 15:39:04 +01:00
Brian Gough
aafa691119 check file exists before running synctex 2017-05-24 10:09:43 +01:00
Brian Gough
a98b2b8032 don't report compile timeouts to sentry
just log them instead
2017-05-24 09:42:05 +01:00
Brian Gough
398ba5ae34 Merge pull request #56 from sharelatex/bg-disable-qpdf-setting
add setting to avoid optimisations outside docker
2017-04-11 14:16:19 +01:00
Brian Gough
a1613eac5a add setting to avoid optimisations outside docker 2017-04-10 16:12:03 +01:00
Brian Gough
3526fde665 Merge pull request #55 from sharelatex/bg-check-pdf-output-is-optimised
use pdfinfo on output to ensure pdfs are optimised
2017-04-10 15:06:22 +01:00
Brian Gough
e1b44beb3f use pdfinfo on output to ensure pdfs are optimised
needed to check that qpdf runs correctly inside the docker container
2017-04-07 11:11:27 +01:00
Brian Gough
17b16dadcd Merge pull request #54 from sharelatex/bg-avoid-running-qpdf-on-already-optimised-files
check if file is optimised before running qpdf
2017-04-05 13:18:32 +01:00
Brian Gough
eb1364f249 check if file is optimised before running qpdf 2017-04-04 16:50:06 +01:00
Shane Kilkelly
834ad57312 Add a .nvmrc file 2017-03-27 14:47:48 +01:00
Brian Gough
19dfaa7d55 Merge pull request #53 from sharelatex/bg-sanitise-paths
additional check for valid rootResource
2017-03-21 13:39:27 +00:00
Brian Gough
b529b8add3 Merge pull request #52 from sharelatex/bg-tikz-externalize
support for tikz externalize
2017-03-21 13:39:14 +00:00
Brian Gough
7ccc9500ed check for \tikzexternalize directly
instead of \usepackage{tikz} and \usepackage{pgf}
2017-03-21 11:36:08 +00:00
Brian Gough
750576d1b0 fix path match 2017-03-21 11:30:32 +00:00
Brian Gough
021d848819 create separate function for path checking 2017-03-21 11:29:37 +00:00
Brian Gough
8803762081 support for tikz externalize
make copy of main file as output.tex for tikz externalize
2017-03-20 10:55:28 +00:00
Brian Gough
5af137f60b additional check for valid rootResource 2017-03-20 10:03:48 +00:00
Brian Gough
f059948e27 update xelatex acceptance test pdf 2017-03-08 11:49:21 +00:00
Brian Gough
7a7c2ee992 improve debugging of failed acceptance tests
use the example name in the output filename
2017-03-08 11:49:12 +00:00
Brian Gough
efe5e22b4c include otf extension in fontawesome test 2017-03-08 11:25:25 +00:00
Shane Kilkelly
03d1936fde Upgrade logger 2017-03-06 14:56:32 +00:00
Shane Kilkelly
a0969ec839 Don't compile acceptance test files during test run 2017-03-06 14:43:14 +00:00
Brian Gough
fdab7763a2 Merge pull request #51 from sharelatex/bg-fix-latexmk-args
allow latexmk to pass through options
2017-03-03 13:19:23 +00:00
Brian Gough
57a5cfa9cb allow latexmk to pass through options
this avoids problems in the latest version of latexmk where the
$pdflatex variable has been replaced by $xelatex and $lualatex when
running with -xelatex or -lualatex
2017-03-02 16:43:35 +00:00
Joe Green
bfb27e6c25 Merge pull request #50 from sharelatex/ho-remove-tcp
remove tcp code, moved to agent load balancer
2017-02-23 14:42:54 +00:00
Henry Oswald
d4d3048719 remove tcp code, moved to agent load balancer 2017-02-23 11:09:18 +00:00
Brian Gough
29594fd0f7 fix acceptance test config file for latex prefix
latex command prefix was in wrong scope
2017-02-21 09:37:05 +00:00
Brian Gough
a50582fd7c add fontawesome acceptance test for xelatex 2017-02-21 09:37:05 +00:00
Henry Oswald
08f0955817 Merge pull request #49 from sharelatex/ho-one-cpu-size
if host has 1 cpu (staging) then set availableWorkingCpus to 1
2017-02-20 15:20:04 +00:00
Henry Oswald
bc1b8f4b2f Update app.coffee 2017-02-20 15:19:04 +00:00
Henry Oswald
599977c3e0 if host has 1 cpu (staging) then set availableWorkingCpus to 1 2017-02-20 15:16:52 +00:00
Brian Gough
071b2269b3 update acceptance tests for reversion to dvipdf 2017-02-13 13:42:44 +00:00
Brian Gough
fde8149579 fix #! in test script 2017-02-09 15:38:25 +00:00
Brian Gough
6b7e33bbc6 show debug info for acceptance tests 2017-02-09 14:17:38 +00:00
Brian Gough
2898a82de8 update acceptance test output for fontawesome 2017-02-07 11:51:21 +00:00
Brian Gough
5b71b849ca added fontawesome acceptance test 2017-02-07 10:00:41 +00:00
Brian Gough
6cb5926c21 fix lualatex require 2017-02-07 08:59:45 +00:00
Brian Gough
3cffb61c74 add luatex85 package to tikz feynman test 2017-02-07 08:49:19 +00:00
Brian Gough
5705455ce1 added acceptance test for tikz-feynman 2017-02-07 08:12:47 +00:00
Brian Gough
71fb15e0ee update knitr_utf acceptance test output
needs to include table of contents from multiple latexmk runs
2017-02-06 16:27:47 +00:00
Brian Gough
819c642b8d add knitr utf8 acceptance test 2017-02-03 15:38:06 +00:00
Brian Gough
20cb52793d add acceptance test for hebrew 2017-02-03 15:16:47 +00:00
Brian Gough
e507bd6394 update acceptance test image for lualatex
small pixel-level change in output
2017-01-31 16:04:59 +00:00
Brian Gough
444b3586a7 increase debugging in acceptance tests 2017-01-31 10:47:49 +00:00
Brian Gough
e25ebd296e add debugging to acceptance tests 2017-01-31 10:40:05 +00:00
Brian Gough
5090ad5c41 update feymp test image
minor pixel change in position of labels in texlive 2016
2017-01-31 10:21:00 +00:00
Brian Gough
bc73f719b2 update asymptote pdf to a4 size for texlive 2016 2017-01-31 09:53:36 +00:00
Brian Gough
d238f73e29 try output.pdf generated with texlive 2016 2017-01-30 15:37:26 +00:00
Brian Gough
ea484da9f4 update latex_compiler test pdf 2017-01-27 12:32:14 +00:00
Brian Gough
b76a81e98b specify papersize explicitly in latex test 2017-01-27 12:21:57 +00:00
Brian Gough
f00be9018d log acceptance test server output to file 2017-01-26 12:20:41 +00:00
Brian Gough
146138f65c try running user as jenkins 2017-01-26 12:06:38 +00:00
Brian Gough
654a43655f update image for docker tests 2017-01-25 14:12:19 +00:00
Brian Gough
b9d6db6caf use local docker image for clsi test 2017-01-25 14:09:44 +00:00
Brian Gough
03e837c1f4 run tests outside container, add settings file 2017-01-25 14:08:39 +00:00
Brian Gough
420db18a03 upgrade to latest sqlite3 2017-01-24 16:06:32 +00:00
Brian Gough
dab92967c8 added docker script for acceptance tests 2017-01-24 12:18:30 +00:00
Brian Gough
0530e21246 fix acceptance tests 2017-01-24 11:07:54 +00:00
Brian Gough
9e53c0b99e fix exception in error log 2016-10-14 10:23:13 +01:00
Shane Kilkelly
61089eca40 Increase memory limit to 64mb 2016-09-28 11:02:58 +01:00
Shane Kilkelly
4827aec30b Add test for new ulimit options 2016-09-23 15:34:29 +01:00
Shane Kilkelly
0900340282 Add CHKTEX_ULIMIT_OPTIONS 2016-09-23 15:32:37 +01:00
James Allen
f7b4883397 Don't delete knitr cache files 2016-09-22 14:14:29 +01:00
James Allen
79b3d2172b Sanitize resource path along with rootResourcePath 2016-09-21 15:09:01 +01:00
Brian Gough
9f49dc8554 Merge pull request #45 from sharelatex/fix-chktex-for-knitr
only run chktex on .tex files, not .Rtex files
2016-09-12 16:36:59 +01:00
Brian Gough
ee170b4e67 only run chktex on .tex files, not .Rtex files
the .tex files produced from knitr have macros which confuse chktex
2016-09-12 16:29:36 +01:00
Shane Kilkelly
47105190be Revert "Revert "Revert "Upgrade to node 4.2"""
This reverts commit 98fb2cab99.
2016-09-01 12:47:13 +01:00
Shane Kilkelly
98fb2cab99 Revert "Revert "Upgrade to node 4.2""
This reverts commit 4128dc6fdd.
2016-09-01 11:22:11 +01:00
Shane Kilkelly
4128dc6fdd Revert "Upgrade to node 4.2"
This reverts commit 8bb12f4d99.
2016-09-01 09:53:12 +01:00
Shane Kilkelly
4a2b2a8707 Merge branch 'master' into sk-node-upgrade 2016-08-31 16:34:25 +01:00
Brian Gough
095e16e953 handle failed compile due to validation error 2016-08-24 15:46:47 +01:00
Brian Gough
3a73971b42 fix commandRunner error to match dockerRunner 2016-08-24 15:45:26 +01:00
Brian Gough
748caeee7d remove chktex error
too many false positives from 'unable to execute latex command'
2016-08-22 15:11:39 +01:00
Brian Gough
cd7ed6ce66 update tests 2016-08-11 10:31:37 +01:00
Brian Gough
2200ac2cf2 capture texcount error output 2016-08-11 10:26:08 +01:00
Brian Gough
928ffc96e6 read wordcount output asynchronously 2016-08-11 09:32:53 +01:00
Brian Gough
ade3da7e0d add missing argument parameter to wordcount call 2016-08-11 09:29:03 +01:00
Brian Gough
e66b1ecdea use a command wrapper for synctex
instead of an alternative child_process object
2016-08-04 16:08:14 +01:00
Brian Gough
c6744caeeb change logging message to be different from LatexRunner 2016-08-04 16:07:36 +01:00
Brian Gough
189648e39a Merge pull request #44 from sharelatex/add-chktex-support
Add chktex support
2016-08-02 14:55:38 +01:00
Brian Gough
8da29e6024 provide setting to override child_process.execFile for synctex 2016-07-29 14:54:24 +01:00
Brian Gough
664e908378 provide validation mode where compilation always exits after chktex 2016-07-27 16:54:27 +01:00
Brian Gough
14837a57ec run chktex when request has check:true 2016-07-26 16:22:38 +01:00
Brian Gough
6524439699 add support for passing additional environment parameters to command runner
includes an example of passing environment variables to chktex
2016-07-26 12:30:29 +01:00
Brian Gough
a7c7f2697f Merge pull request #43 from sharelatex/stop-compile
add support for stopping compile
2016-07-18 11:16:53 +01:00
Brian Gough
fdf274fb82 remove dead code 2016-07-18 11:05:45 +01:00
Brian Gough
69666bef60 add support for stopping compile 2016-07-14 16:43:52 +01:00
Henry Oswald
cd8e60195c Merge pull request #42 from WaeCo/patch-1
Set default project_cache_length_ms to 1 day
2016-07-13 21:32:02 +01:00
WaeCo
d6808c11cc Set default project_cache_length_ms to 1 day
`project_cache_length_ms` was only `60*60*24 = 1.5 min` which is a little bit short. Default of one day seams more reasonable.
2016-07-13 13:26:32 -07:00
Brian Gough
133f522e7b Merge pull request #41 from sharelatex/per-user-containers-part-3
Reduce number of cached builds for per-user containers
2016-06-30 08:06:01 +01:00
Brian Gough
d29416fc77 keep one extra build until per-page pdf serving is enabled 2016-06-29 16:31:16 +01:00
Brian Gough
c486d6c215 only keep a single cached output directory in per-user containers 2016-06-28 09:28:40 +01:00
Shane Kilkelly
8bb12f4d99 Upgrade to node 4.2 2016-06-20 09:31:30 +01:00
Shane Kilkelly
e4ffc94de8 Move the latexmk timing command into a configurable latexmkCommandPrefix.
By default, no timing information will be taken.
On Linux with GNU user land, this value should be configured to `["/usr/bin/time", "-v"]`.
On Mac, gnu-time should be installed and configured to `["/usr/local/bin/gtime", "-v"]`.
2016-06-17 14:38:08 +01:00
Brian Gough
0b8435e358 add route to serve files from top level of per user containers 2016-06-15 16:12:19 +01:00
Brian Gough
801f09e7ed Merge branch 'per-user-containers-part-2'
Conflicts:
	app/coffee/CompileController.coffee
2016-06-13 09:33:41 +01:00
Brian Gough
603b3d617c Merge pull request #39 from sharelatex/per-user-containers-part-1
Per user containers part 1
2016-06-09 15:17:35 +01:00
Henry Oswald
b97627d6d8 use process id so link process to smoke test 2016-06-07 14:47:51 +01:00
Henry Oswald
da02661d53 add random string to smoke tests to avoid collision 2016-06-07 14:39:01 +01:00
Brian Gough
6e017ecaf1 log user_id when clearing project 2016-06-02 15:32:33 +01:00
Brian Gough
0887fe3a72 add per-user routes for clearing cache and extend expiry methods
this adds separate functionality for clearing the cache (assets and
database) and the project compile directory for a specific user
2016-06-02 15:32:33 +01:00
Brian Gough
226e6c87b1 add per-user routes and methods 2016-06-02 15:32:31 +01:00
Brian Gough
8c42a353e1 put the build id in the output file urls
the url attribute will now give the preferred location for accessing
the output file, without the url having to be constructed by the web
client
2016-06-02 15:30:50 +01:00
Brian Gough
78b88683fc put the build id in the output file urls
the url attribute will now give the preferred location for accessing
the output file, without the url having to be constructed by the web
client
2016-06-02 15:29:56 +01:00
Henry Oswald
ac3b7a571a log out error on synctex 2016-05-27 16:18:18 +01:00
Henry Oswald
cda1e301f6 log out errors more clearly 2016-05-27 14:45:39 +01:00
Henry Oswald
da324a8dd0 added logger.info to test setup 2016-05-24 14:12:02 +01:00
Henry Oswald
b2f687c061 log out which command logger is used 2016-05-24 14:08:39 +01:00
Henry Oswald
2c3b1126b0 log out if the command running is being used 2016-05-23 15:45:39 +01:00
Henry Oswald
22f730c3e9 parallelFileDownloads defaults to 1, sql can't take it 2016-05-23 14:31:27 +01:00
Henry Oswald
2e97bcba3a add error handler on CommandRunner 2016-05-23 14:13:55 +01:00
Brian Gough
0da85d5d03 be ready to serve files from per-user containers 2016-05-20 10:23:07 +01:00
Brian Gough
3379577499 fix error in log for expiry timeout 2016-05-20 10:23:07 +01:00
Henry Oswald
855169b571 Merge branch 'master' of https://github.com/sharelatex/clsi-sharelatex 2016-05-19 16:57:19 +01:00
Henry Oswald
6b107bd20a log out EXPIRY_TIMEOUT 2016-05-19 16:57:14 +01:00
Henry Oswald
a2c2fc3a51 make cached assets ttl set via config 2016-05-19 16:51:50 +01:00
Brian Gough
f8ae215c1e avoid clobbering the existing port variable 2016-05-19 16:38:18 +01:00
Brian Gough
d26c6b933e return the file path in the output file list for easy lookup 2016-05-19 16:38:18 +01:00
Brian Gough
4496ddddfd Merge pull request #38 from sharelatex/add-fast-path-to-pdf
Add fast path to pdf
2016-05-13 12:32:26 +01:00
Brian Gough
434e00cb74 make the build id a secure random token
we allow existing build ids to work for backwards compatibility
this can be removed after some time
2016-05-13 10:11:35 +01:00
Brian Gough
f92c70935b allow direct path to output file /project/project_id/build/build_id/output/*
this avoids use of the query string ?build=... and so we can match the
url directly with the nginx location directive
2016-05-13 10:10:48 +01:00
Brian Gough
51f87c5f79 fix logic excluding smoke test in metric 2016-05-10 10:10:01 +01:00
Brian Gough
143913c67f fix tagname for graphite 2016-05-10 09:41:39 +01:00
Brian Gough
dfd2bc31ef record system time 2016-05-10 09:12:13 +01:00
Brian Gough
e70bd3ae8e preserve existing metric name 2016-05-10 09:12:00 +01:00
Brian Gough
0a5ca6b0fa add timing information from /usr/bin/time 2016-05-09 16:00:24 +01:00
Brian Gough
834668b033 add a metric for the TeXLive image used on each compile 2016-05-09 15:36:11 +01:00
Henry Oswald
35240fbd4d move back to 2.5 days cache for moment 2016-04-21 17:40:09 +01:00
Henry Oswald
5f7cd5ece5 added project status endpoint
used for getting the server a project is on
2016-04-20 15:38:05 +01:00
Henry Oswald
6860d2be6c increased clsi cache to 3.5 days 2016-04-13 09:29:57 +01:00
Henry Oswald
3c021fd4c9 ignore ECONNRESET 2016-04-12 13:32:58 +01:00
Henry Oswald
f453f954e4 use socket.end for tcp checks 2016-04-12 10:49:45 +01:00
Henry Oswald
cd499fa4e5 server load endpoint uses settings for port 2016-04-11 13:47:06 +01:00
Henry Oswald
7799e0bfdd return 0 for server which is being hammered
socket.destroy when finished
2016-04-08 15:40:02 +01:00
Henry Oswald
6ca8c10734 added err handler to socket 2016-04-08 15:25:00 +01:00
Henry Oswald
84cba7365f work of 1 min load and set server as up 2016-04-08 15:18:22 +01:00
Henry Oswald
11be12fc8e evaluate on every call 2016-04-08 14:14:05 +01:00
Henry Oswald
3e70c0f8e4 added example server load tcp server 2016-04-08 13:31:23 +01:00
Brian Gough
558e9ae22b don't log errors when files have disappeared from build directory 2016-04-07 16:16:39 +01:00
Brian Gough
83e373d7e1 log errors in detail when file cannot be removed 2016-04-04 16:22:48 +01:00
Brian Gough
24fc9391c3 upgrade to the latest version of request 2016-03-31 14:03:48 +01:00
Brian Gough
7ff56c4793 suppress error when removing nonexistent file from cache 2016-03-31 13:33:42 +01:00
Brian Gough
665dbff75a parameter check on project_id 2016-03-31 12:12:25 +01:00
Brian Gough
5d6fb4579a remove console.log 2016-03-31 11:59:17 +01:00
Brian Gough
bd036534e5 check directory exists before attempting to clear it 2016-03-31 11:59:17 +01:00
Brian Gough
3dcd4af62e always create project directory when syncing resources to disk
avoids errors when project is empty
2016-03-31 11:59:17 +01:00
Brian Gough
fe46a96fd2 don't log missing files as warnings, but do report file access errors 2016-03-31 11:14:39 +01:00
Brian Gough
8fcbec5c0f add support for sentry 2016-03-30 14:35:47 +01:00
James Allen
fbb00ebf2f Only archive main log and blg 2016-03-30 14:10:07 +01:00
James Allen
6117cac1fd Ignore both .cache and .archive and other hidden files in finding output files 2016-03-30 11:41:11 +01:00
James Allen
d949d4ac32 Don't timestamp strace logs otherwise it runs as anew container each time since the command changes 2016-03-30 10:59:01 +01:00
James Allen
6af22cf184 Add in flags to run strace and capture logs 2016-03-30 10:37:22 +01:00
Brian Gough
9f104a4f57 bugfix - avoid double counting compiles 2016-03-17 14:37:34 +00:00
Brian Gough
595bfe09ac add metric for qpdf 2016-03-17 09:55:18 +00:00
Brian Gough
e64b08fcbe add metrics for latexmk runs and errors 2016-03-17 09:55:18 +00:00
Henry Oswald
dcfe1118d4 increased EXPIRY_TIMEOUT from 1.5 days to 2.5 days 2016-03-10 10:30:37 +00:00
James Allen
89acd36dde Send .svg files as text/plain to prevent executable JS if they are loaded as SVG in the browser 2016-03-10 09:32:32 +00:00
James Allen
a3383f11a1 Make draft mode regex global 2016-02-02 15:28:59 +00:00
James Allen
2df886e330 Remove left over debug log line 2016-02-02 14:28:51 +00:00
James Allen
d96605d5e8 Inject [draft] option to documentclass if draft option is passed 2016-02-02 14:26:14 +00:00
James Allen
03b75b12cf Download up to 5 files in parallel 2016-02-01 13:19:16 +00:00
James Allen
86cf05c732 Support configurable images in wordcount end point 2016-01-19 14:12:41 +00:00
James Allen
4497352a3a Allow optional image name to be passed 2016-01-15 09:59:06 +00:00
Henry Oswald
601a3e4805 Merge branch 'master' of https://github.com/sharelatex/clsi-sharelatex 2015-12-15 19:34:34 +00:00
Henry Oswald
0ea28710f5 fixed missing value in logger 2015-12-15 19:33:37 +00:00
James Allen
2b5e7be964 Remove undefined reference to dst 2015-12-03 14:54:48 +00:00
Henry Oswald
c178458223 added try catch around word count where a file is not created 2015-11-12 15:19:22 +00:00
Henry Oswald
3ed29b3489 increased cache time to 1.5 days 2015-10-21 10:02:30 +01:00
Shane Kilkelly
29be2dc700 When serving output files, intelligently determine the appropriate content-type.
cherry pick 6fa3fda3ed28239cf3ac9720629f9707663aa197 from datajoy.
2015-09-21 16:59:35 +01:00
Henry Oswald
d83efdbc98 lock down smoke test and metrics version 2015-09-17 10:30:12 +01:00
Henry Oswald
f6af22879f Merge pull request #33 from sharelatex/pr/28
wordcount
2015-09-09 14:03:20 +01:00
Henry Oswald
74c393cda3 - fixed bug with texcount returning wrong data for nauty lines
- improved acceptence test for word count to use nauty lines
2015-09-09 13:52:45 +01:00
Henry Oswald
561eaa0d66 add -inc to word count
use -inc to word count included files

also moved private function to bottom
2015-09-09 12:47:08 +01:00
Henry Oswald
440d1c605f added test to check compile should continue on error downloading http
resource

also improved logging
2015-09-09 09:44:38 +01:00
Henry Oswald
6fae6ff40c don't error if a http resource can not be download
try and continue, log the error but you might still be able to
compile. prevents issue with badly uploaded images in filstore
2015-09-08 21:58:27 +01:00
Henrique Dias
caef254b80 move texcount to docker 2015-09-08 10:19:46 -03:00
Henrique Dias
ba475db052 Merge remote-tracking branch 'upstream/master' into texcount 2015-09-08 09:13:04 -03:00
Brian Gough
c2054a5ec3 add memory logger from metrics-sharelatex 2015-08-14 14:47:42 +01:00
Henry Oswald
f270ff9801 Merge branch 'master' of https://github.com/sharelatex/clsi-sharelatex 2015-06-12 17:11:11 +01:00
Henry Oswald
71d2427cf1 added some load tests in 2015-06-12 17:11:03 +01:00
Henrique Dias
a04d951b40 add unit test 2015-06-08 19:27:47 -03:00
Henrique Dias
25d73ab6f9 initial version of texcount 2015-06-08 18:35:24 -03:00
Brian Gough
f707ad0926 Merge pull request #23 from sharelatex/add-mysql-indexes
add indexes to mysql db
2015-05-21 12:21:59 +01:00
Brian Gough
779d79ebfd Merge pull request #22 from sharelatex/upgrade-sequelize
upgrade sequelize and mysql
2015-05-21 12:21:50 +01:00
Brian Gough
03c516a87e Merge pull request #21 from sharelatex/fix-urlfetcher-streams
clean up error handling in UrlFetcher
2015-05-21 12:21:31 +01:00
Brian Gough
19993f4548 Merge pull request #20 from sharelatex/fix-deprecated-express-calls
replace deprecated express send(code,body) calls
2015-05-21 12:21:09 +01:00
Brian Gough
ff5b203ecf added comments 2015-05-21 11:33:13 +01:00
Brian Gough
7d8bc9fed9 upgrade sequelize and mysql 2015-05-15 16:58:27 +01:00
Brian Gough
484c0da84f add indexes to db 2015-05-15 16:28:35 +01:00
Brian Gough
3c97b98fc4 clean up error handling in UrlFetcher 2015-05-15 16:28:11 +01:00
Brian Gough
03c105c3d9 replace deprecated send(code,body) calls 2015-05-15 16:27:39 +01:00
Brian Gough
889fa65d0c clean up stream handling for file copy 2015-05-15 16:25:49 +01:00
Brian Gough
558b5e000f only run qpdf for the main output.pdf file
was previously matching any pdf file, which caused it to run for
embedded pdf figures produced during the mklatex run
2015-05-15 16:23:58 +01:00
Henry Oswald
94c4187eb2 change regex checking file request ensure other files can not be accessed 2015-05-12 15:18:09 +01:00
Brian Gough
9d3fdcf8b4 additional validation of requests 2015-05-11 12:10:13 +01:00
Brian Gough
58ecfa69e6 use the latest versions of metrics and smoketest modules 2015-05-05 10:54:59 +01:00
Brian Gough
d90fe49a4e avoid leak when calling chai.should() repeatedly in smoke test 2015-05-05 10:03:17 +01:00
Brian Gough
2e91868bc8 disable sequelize logging by default
prevent any leaking of objects to console during debugging
2015-05-05 10:03:17 +01:00
Henry Oswald
d9d16b7189 make startup message consistent 2015-04-30 15:07:48 +01:00
Brian Gough
fd6386207b remove debugging from tests 2015-04-29 16:05:49 +01:00
Brian Gough
9892751ff6 prevent leak of urlStream on failed downloads 2015-04-29 16:05:49 +01:00
Brian Gough
651279b21f log errors when downloading files and clean up failed downloads 2015-04-29 16:05:49 +01:00
Brian Gough
8db907c766 invalidate the cache if there is an error copying a file 2015-04-29 15:54:38 +01:00
Brian Gough
85c6c3fe2b log errors when copying files from cache 2015-04-29 15:54:08 +01:00
Brian Gough
143f948193 add heapdump support for memory profiling 2015-04-09 14:40:02 +01:00
James Allen
a35df6d829 Release version 0.1.4 2015-03-20 15:25:02 +00:00
Brian Gough
1a794d804a Merge branch 'master' of github.com:sharelatex/clsi-sharelatex 2015-03-16 16:47:56 +00:00
Brian Gough
24e20a79f4 remove unnecessary call to async.series in OutputFileFinder
callback was previously async but is now synchronous, so high stack
usage.
2015-03-16 16:47:25 +00:00
Brian Gough
c47f49e24b Merge pull request #17 from sharelatex/add-v8-performance
add v8 profiler
2015-03-16 15:18:48 +00:00
Brian Gough
65f2f23cf6 add v8 profiler on /profile?time=MS url 2015-03-16 15:02:45 +00:00
Brian Gough
8d7d637eed Merge pull request #16 from sharelatex/cache-output-files
Keep output files in cache
2015-03-02 12:06:06 +00:00
Brian Gough
7551bc3135 reduce cache limit for pdfs 2015-03-02 11:31:48 +00:00
Brian Gough
75ef0d6581 skip cache directory error when empty 2015-03-02 09:58:20 +00:00
Brian Gough
31f62c7a7b Merge branch 'master' into cache-output-files 2015-03-02 09:18:44 +00:00
Brian Gough
3a4dd9df50 fix double callback for proc.on 'error' and proc.on 'close' 2015-02-27 16:07:02 +00:00
Brian Gough
916b4cb40b move convert tests from middleware to restricted static server 2015-02-27 15:38:57 +00:00
Brian Gough
37cc9f3715 provide a static server which forbids symlinks
prevents mismatch between rootdir of server and rootdir of symlink
checking middleware
2015-02-27 13:57:57 +00:00
Brian Gough
0692e964ef use OutputCacheManager to construct static path to files 2015-02-27 13:16:01 +00:00
Brian Gough
198e1ef492 cleanup and logging 2015-02-27 13:15:35 +00:00
Brian Gough
280d64cf60 remove debugging code 2015-02-26 15:32:01 +00:00
Brian Gough
e7ed8d786a fix tests to allow for build parameter 2015-02-26 15:31:12 +00:00
James Allen
81e85de169 Release version 0.1.3 2015-02-26 11:20:56 +00:00
Brian Gough
151ea99639 accept build id parameter when serving static files 2015-02-25 17:05:19 +00:00
Brian Gough
b8cdd4fa85 added package dependencies for caching 2015-02-24 16:09:55 +00:00
Brian Gough
163a33674b add an optimisation pass for the cached output files 2015-02-24 15:48:34 +00:00
Brian Gough
67bfeacab8 skip the cache directory when finding output files 2015-02-24 14:40:22 +00:00
Brian Gough
1923352e66 save output files in a .cache directory 2015-02-24 14:40:05 +00:00
Brian Gough
f37004cec6 update sanitizePath regex
remove accidental inclusion of , and add null char \x00
2015-02-13 11:28:43 +00:00
James Allen
1a7500f102 Allow non-latin characters in the rootResourcePath 2015-02-13 11:21:35 +00:00
James Allen
90cda12ed9 Sanitize rootResourcePath 2015-02-11 16:39:43 +00:00
James Allen
c84bd4fa3f Release version 0.1.2 2015-02-10 13:19:42 +00:00
James Allen
84f3d3061d Don't return error if directory doesn't exist yet 2014-12-09 11:25:23 +00:00
James Allen
2c4fbd10ed Add in some debugging logging 2014-12-09 11:16:16 +00:00
James Allen
ff94a76eb9 Use find -type f to get a list of output files 2014-12-09 11:08:07 +00:00
Henry Oswald
92338ab419 replaced old symlink logic with tested middlewear based on fs.realpath 2014-12-04 23:54:22 +00:00
James Allen
5b2031b84f Check file is not a symlink before returning it 2014-12-04 22:07:37 +00:00
James Allen
94397854c6 Add in missing error check 2014-12-04 21:37:09 +00:00
Brian Gough
6bf8c22d78 send a strong etag for the output.pdf file, needed for byte ranges in pdf.js 2014-12-02 14:30:24 +00:00
Henry Oswald
b4f0da0c42 err != error 2014-11-27 16:19:01 +00:00
Henry Oswald
4886620d8a Merge branch 'master' of https://github.com/sharelatex/clsi-sharelatex 2014-11-27 16:11:11 +00:00
Henry Oswald
fc674370bd respect the status code on the error if it exists 2014-11-27 16:11:00 +00:00
James Allen
b418ea201b Update acceptance tests for new knitr, and remove markdown 2014-10-29 10:59:32 +00:00
James Allen
7f9c9176a9 Force mimetype of output files to be safe 2014-10-28 12:07:26 +00:00
Henry Oswald
af86745112 increase max compile to 4 mins 2014-10-17 11:03:08 +01:00
Henry Oswald
22e8ee59af Merge branch 'master' of https://github.com/sharelatex/clsi-sharelatex 2014-10-17 10:22:27 +01:00
Henry Oswald
225a12fcd2 up timeout to 6 mins 2014-10-17 10:14:23 +01:00
77 changed files with 3891 additions and 378 deletions

1
.nvmrc Normal file
View File

@@ -0,0 +1 @@
6.11.2

View File

@@ -1,8 +1,5 @@
language: node_js
node_js:
- "0.10"
before_install:
- npm install -g grunt-cli

View File

@@ -46,10 +46,16 @@ module.exports = (grunt) ->
app:
src: "app.js"
mkdir:
all:
options:
create: ["cache", "compiles"]
mochaTest:
unit:
options:
reporter: "spec"
grep: grunt.option("grep")
src: ["test/unit/js/**/*.js"]
acceptance:
options:
@@ -69,6 +75,7 @@ module.exports = (grunt) ->
grunt.loadNpmTasks 'grunt-shell'
grunt.loadNpmTasks 'grunt-execute'
grunt.loadNpmTasks 'grunt-bunyan'
grunt.loadNpmTasks 'grunt-mkdir'
grunt.registerTask 'compile:bin', () ->
callback = @async()
@@ -92,6 +99,6 @@ module.exports = (grunt) ->
grunt.registerTask 'install', 'compile:app'
grunt.registerTask 'default', ['run']
grunt.registerTask 'default', ['mkdir', 'run']

101
Jenkinsfile vendored Normal file
View File

@@ -0,0 +1,101 @@
pipeline {
agent any
triggers {
pollSCM('* * * * *')
cron('@daily')
}
stages {
stage('Clean') {
steps {
// This is a terrible hack to set the file ownership to jenkins:jenkins so we can cleanup the directory
sh 'docker run -v $(pwd):/app --rm busybox /bin/chown -R 111:119 /app'
sh 'rm -fr node_modules'
}
}
stage('Install') {
agent {
docker {
image 'node:6.11.2'
args "-v /var/lib/jenkins/.npm:/tmp/.npm -e HOME=/tmp"
reuseNode true
}
}
steps {
sh 'git config --global core.logallrefupdates false'
sh 'rm -fr node_modules'
checkout([$class: 'GitSCM', branches: [[name: '*/master']], extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: '_docker-runner'], [$class: 'CloneOption', shallow: true]], userRemoteConfigs: [[credentialsId: 'GIT_DEPLOY_KEY', url: 'git@github.com:sharelatex/docker-runner-sharelatex']]])
sh 'npm install ./_docker-runner'
sh 'rm -fr ./_docker-runner ./_docker-runner@tmp'
sh 'npm install'
sh 'npm rebuild'
sh 'npm install --quiet grunt-cli'
}
}
stage('Compile and Test') {
agent {
docker {
image 'node:6.11.2'
reuseNode true
}
}
steps {
sh 'node_modules/.bin/grunt compile:app'
sh 'node_modules/.bin/grunt compile:acceptance_tests'
sh 'NODE_ENV=development node_modules/.bin/grunt test:unit'
}
}
stage('Acceptance Tests') {
environment {
TEXLIVE_IMAGE="quay.io/sharelatex/texlive-full:2017.1"
}
steps {
sh 'mkdir -p compiles cache'
// Not yet running, due to volumes/sibling containers
sh 'docker container prune -f'
sh 'docker pull $TEXLIVE_IMAGE'
sh 'docker pull sharelatex/acceptance-test-runner:clsi-6.11.2'
sh 'docker run --rm -e SIBLING_CONTAINER_USER=root -e SANDBOXED_COMPILES_HOST_DIR=$(pwd)/compiles -e SANDBOXED_COMPILES_SIBLING_CONTAINERS=true -e TEXLIVE_IMAGE=$TEXLIVE_IMAGE -v /var/run/docker.sock:/var/run/docker.sock -v $(pwd):/app sharelatex/acceptance-test-runner:clsi-6.11.2'
// This is a terrible hack to set the file ownership to jenkins:jenkins so we can cleanup the directory
sh 'docker run -v $(pwd):/app --rm busybox /bin/chown -R 111:119 /app'
sh 'rm -r compiles cache server.log db.sqlite config/settings.defaults.coffee'
}
}
stage('Package') {
steps {
sh 'echo ${BUILD_NUMBER} > build_number.txt'
sh 'touch build.tar.gz' // Avoid tar warning about files changing during read
sh 'tar -czf build.tar.gz --exclude=build.tar.gz --exclude-vcs .'
}
}
stage('Publish') {
steps {
withAWS(credentials:'S3_CI_BUILDS_AWS_KEYS', region:"${S3_REGION_BUILD_ARTEFACTS}") {
s3Upload(file:'build.tar.gz', bucket:"${S3_BUCKET_BUILD_ARTEFACTS}", path:"${JOB_NAME}/${BUILD_NUMBER}.tar.gz")
// The deployment process uses this file to figure out the latest build
s3Upload(file:'build_number.txt', bucket:"${S3_BUCKET_BUILD_ARTEFACTS}", path:"${JOB_NAME}/latest")
}
}
}
}
post {
failure {
mail(from: "${EMAIL_ALERT_FROM}",
to: "${EMAIL_ALERT_TO}",
subject: "Jenkins build failed: ${JOB_NAME}:${BUILD_NUMBER}",
body: "Build: ${BUILD_URL}")
}
}
// The options directive is for configuration that applies to the whole job.
options {
// we'd like to make sure remove old builds, so we don't fill up our storage!
buildDiscarder(logRotator(numToKeepStr:'50'))
// And we'd really like to be sure that this build doesn't hang forever, so let's time it out after:
timeout(time: 30, unit: 'MINUTES')
}
}

View File

@@ -2,13 +2,23 @@ CompileController = require "./app/js/CompileController"
Settings = require "settings-sharelatex"
logger = require "logger-sharelatex"
logger.initialize("clsi")
if Settings.sentry?.dsn?
logger.initializeErrorReporting(Settings.sentry.dsn)
smokeTest = require "smoke-test-sharelatex"
ContentTypeMapper = require "./app/js/ContentTypeMapper"
Errors = require './app/js/Errors'
Path = require "path"
fs = require "fs"
Metrics = require "metrics-sharelatex"
Metrics.initialize("clsi")
Metrics.open_sockets.monitor(logger)
Metrics.memory.monitor(logger)
ProjectPersistenceManager = require "./app/js/ProjectPersistenceManager"
OutputCacheManager = require "./app/js/OutputCacheManager"
require("./app/js/db").sync()
@@ -21,23 +31,92 @@ app.use Metrics.http.monitor(logger)
# Compile requests can take longer than the default two
# minutes (including file download time), so bump up the
# timeout a bit.
TIMEOUT = threeMinutes = 3 * 60 * 1000
TIMEOUT = 6 * 60 * 1000
app.use (req, res, next) ->
req.setTimeout TIMEOUT
res.setTimeout TIMEOUT
next()
app.param 'project_id', (req, res, next, project_id) ->
if project_id?.match /^[a-zA-Z0-9_-]+$/
next()
else
next new Error("invalid project id")
app.param 'user_id', (req, res, next, user_id) ->
if user_id?.match /^[0-9a-f]{24}$/
next()
else
next new Error("invalid user id")
app.param 'build_id', (req, res, next, build_id) ->
if build_id?.match OutputCacheManager.BUILD_REGEX
next()
else
next new Error("invalid build id #{build_id}")
app.post "/project/:project_id/compile", bodyParser.json(limit: "5mb"), CompileController.compile
app.post "/project/:project_id/compile/stop", CompileController.stopCompile
app.delete "/project/:project_id", CompileController.clearCache
app.get "/project/:project_id/sync/code", CompileController.syncFromCode
app.get "/project/:project_id/sync/pdf", CompileController.syncFromPdf
app.get "/project/:project_id/wordcount", CompileController.wordcount
app.get "/project/:project_id/status", CompileController.status
staticServer = express.static(Settings.path.compilesDir)
app.get "/project/:project_id/output/*", (req, res, next) ->
req.url = "/#{req.params.project_id}/#{req.params[0]}"
# Per-user containers
app.post "/project/:project_id/user/:user_id/compile", bodyParser.json(limit: "5mb"), CompileController.compile
app.post "/project/:project_id/user/:user_id/compile/stop", CompileController.stopCompile
app.delete "/project/:project_id/user/:user_id", CompileController.clearCache
app.get "/project/:project_id/user/:user_id/sync/code", CompileController.syncFromCode
app.get "/project/:project_id/user/:user_id/sync/pdf", CompileController.syncFromPdf
app.get "/project/:project_id/user/:user_id/wordcount", CompileController.wordcount
ForbidSymlinks = require "./app/js/StaticServerForbidSymlinks"
# create a static server which does not allow access to any symlinks
# avoids possible mismatch of root directory between middleware check
# and serving the files
staticServer = ForbidSymlinks express.static, Settings.path.compilesDir, setHeaders: (res, path, stat) ->
if Path.basename(path) == "output.pdf"
# Calculate an etag in the same way as nginx
# https://github.com/tj/send/issues/65
etag = (path, stat) ->
'"' + Math.ceil(+stat.mtime / 1000).toString(16) +
'-' + Number(stat.size).toString(16) + '"'
res.set("Etag", etag(path, stat))
res.set("Content-Type", ContentTypeMapper.map(path))
app.get "/project/:project_id/user/:user_id/build/:build_id/output/*", (req, res, next) ->
# for specific build get the path from the OutputCacheManager (e.g. .clsi/buildId)
req.url = "/#{req.params.project_id}-#{req.params.user_id}/" + OutputCacheManager.path(req.params.build_id, "/#{req.params[0]}")
staticServer(req, res, next)
app.get "/project/:project_id/build/:build_id/output/*", (req, res, next) ->
# for specific build get the path from the OutputCacheManager (e.g. .clsi/buildId)
req.url = "/#{req.params.project_id}/" + OutputCacheManager.path(req.params.build_id, "/#{req.params[0]}")
staticServer(req, res, next)
app.get "/project/:project_id/user/:user_id/output/*", (req, res, next) ->
# for specific user get the path to the top level file
req.url = "/#{req.params.project_id}-#{req.params.user_id}/#{req.params[0]}"
staticServer(req, res, next)
app.get "/project/:project_id/output/*", (req, res, next) ->
if req.query?.build? && req.query.build.match(OutputCacheManager.BUILD_REGEX)
# for specific build get the path from the OutputCacheManager (e.g. .clsi/buildId)
req.url = "/#{req.params.project_id}/" + OutputCacheManager.path(req.query.build, "/#{req.params[0]}")
else
req.url = "/#{req.params.project_id}/#{req.params[0]}"
staticServer(req, res, next)
app.get "/oops", (req, res, next) ->
logger.error {err: "hello"}, "test error"
res.send "error\n"
app.get "/status", (req, res, next) ->
res.send "CLSI is alive\n"
@@ -54,19 +133,37 @@ if Settings.smokeTest
do runSmokeTest = ->
logger.log("running smoke tests")
smokeTest.run(require.resolve(__dirname + "/test/smoke/js/SmokeTests.js"))({}, resCacher)
setTimeout(runSmokeTest, 20 * 1000)
setTimeout(runSmokeTest, 30 * 1000)
app.get "/health_check", (req, res)->
res.contentType(resCacher?.setContentType)
res.send resCacher?.code, resCacher?.body
res.status(resCacher?.code).send(resCacher?.body)
profiler = require "v8-profiler"
app.get "/profile", (req, res) ->
time = parseInt(req.query.time || "1000")
profiler.startProfiling("test")
setTimeout () ->
profile = profiler.stopProfiling("test")
res.json(profile)
, time
app.get "/heapdump", (req, res)->
require('heapdump').writeSnapshot '/tmp/' + Date.now() + '.clsi.heapsnapshot', (err, filename)->
res.send filename
app.use (error, req, res, next) ->
logger.error err: error, "server error"
res.send 500
if error instanceof Errors.NotFoundError
logger.warn {err: error, url: req.url}, "not found error"
return res.sendStatus(404)
else
logger.error {err: error, url: req.url}, "server error"
res.sendStatus(error?.statusCode || 500)
app.listen port = (Settings.internal?.clsi?.port or 3013), host = (Settings.internal?.clsi?.host or "localhost"), (error) ->
logger.log "CLSI listening on #{host}:#{port}"
logger.info "CLSI starting up, listening on #{host}:#{port}"
setInterval () ->
ProjectPersistenceManager.clearExpiredProjects()
, tenMinutes = 10 * 60 * 1000

View File

@@ -1,12 +1,44 @@
spawn = require("child_process").spawn
logger = require "logger-sharelatex"
logger.info "using standard command runner"
module.exports = CommandRunner =
run: (project_id, command, directory, timeout, callback = (error) ->) ->
run: (project_id, command, directory, image, timeout, environment, callback = (error) ->) ->
command = (arg.replace('$COMPILE_DIR', directory) for arg in command)
logger.log project_id: project_id, command: command, directory: directory, "running command"
logger.warn "timeouts and sandboxing are not enabled with CommandRunner"
proc = spawn command[0], command.slice(1), stdio: "inherit", cwd: directory
proc.on "close", () ->
callback()
# merge environment settings
env = {}
env[key] = value for key, value of process.env
env[key] = value for key, value of environment
# run command as detached process so it has its own process group (which can be killed if needed)
proc = spawn command[0], command.slice(1), stdio: "inherit", cwd: directory, detached: true, env: env
proc.on "error", (err)->
logger.err err:err, project_id:project_id, command: command, directory: directory, "error running command"
callback(err)
proc.on "close", (code, signal) ->
logger.info code:code, signal:signal, project_id:project_id, "command exited"
if signal is 'SIGTERM' # signal from kill method below
err = new Error("terminated")
err.terminated = true
return callback(err)
else if code is 1 # exit status from chktex
err = new Error("exited")
err.code = code
return callback(err)
else
callback()
return proc.pid # return process id to allow job to be killed if necessary
kill: (pid, callback = (error) ->) ->
try
process.kill -pid # kill all processes in group
catch err
return callback(err)
callback()

View File

@@ -4,6 +4,7 @@ Settings = require "settings-sharelatex"
Metrics = require "./Metrics"
ProjectPersistenceManager = require "./ProjectPersistenceManager"
logger = require "logger-sharelatex"
Errors = require "./Errors"
module.exports = CompileController =
compile: (req, res, next = (error) ->) ->
@@ -11,44 +12,75 @@ module.exports = CompileController =
RequestParser.parse req.body, (error, request) ->
return next(error) if error?
request.project_id = req.params.project_id
request.user_id = req.params.user_id if req.params.user_id?
ProjectPersistenceManager.markProjectAsJustAccessed request.project_id, (error) ->
return next(error) if error?
CompileManager.doCompile request, (error, outputFiles = []) ->
if error?
logger.error err: error, project_id: request.project_id, "error running compile"
CompileManager.doCompileWithLock request, (error, outputFiles = []) ->
if error instanceof Errors.AlreadyCompilingError
code = 423 # Http 423 Locked
status = "compile-in-progress"
else if error instanceof Errors.FilesOutOfSyncError
code = 409 # Http 409 Conflict
status = "retry"
else if error?.terminated
status = "terminated"
else if error?.validate
status = "validation-#{error.validate}"
else if error?
if error.timedout
status = "timedout"
logger.log err: error, project_id: request.project_id, "timeout running compile"
else
status = "error"
code = 500
logger.error err: error, project_id: request.project_id, "error running compile"
else
status = "failure"
for file in outputFiles
if file.path?.match(/output\.pdf$/)
status = "success"
if file.path?.match(/output\.html$/)
status = "success"
# log an error if any core files are found
for file in outputFiles
if file.path is "core"
logger.error project_id:request.project_id, req:req, outputFiles:outputFiles, "core file found in output"
timer.done()
res.send (code or 200), {
res.status(code or 200).send {
compile:
status: status
error: error?.message or error
error: error?.message or error
outputFiles: outputFiles.map (file) ->
url: "#{Settings.apis.clsi.url}/project/#{request.project_id}/output/#{file.path}"
url:
"#{Settings.apis.clsi.url}/project/#{request.project_id}" +
(if request.user_id? then "/user/#{request.user_id}" else "") +
(if file.build? then "/build/#{file.build}" else "") +
"/output/#{file.path}"
path: file.path
type: file.type
build: file.build
}
clearCache: (req, res, next = (error) ->) ->
ProjectPersistenceManager.clearProject req.params.project_id, (error) ->
stopCompile: (req, res, next) ->
{project_id, user_id} = req.params
CompileManager.stopCompile project_id, user_id, (error) ->
return next(error) if error?
res.send 204 # No content
res.sendStatus(204)
clearCache: (req, res, next = (error) ->) ->
ProjectPersistenceManager.clearProject req.params.project_id, req.params.user_id, (error) ->
return next(error) if error?
res.sendStatus(204) # No content
syncFromCode: (req, res, next = (error) ->) ->
file = req.query.file
line = parseInt(req.query.line, 10)
column = parseInt(req.query.column, 10)
project_id = req.params.project_id
user_id = req.params.user_id
CompileManager.syncFromCode project_id, file, line, column, (error, pdfPositions) ->
CompileManager.syncFromCode project_id, user_id, file, line, column, (error, pdfPositions) ->
return next(error) if error?
res.send JSON.stringify {
pdf: pdfPositions
@@ -59,9 +91,27 @@ module.exports = CompileController =
h = parseFloat(req.query.h)
v = parseFloat(req.query.v)
project_id = req.params.project_id
user_id = req.params.user_id
CompileManager.syncFromPdf project_id, page, h, v, (error, codePositions) ->
CompileManager.syncFromPdf project_id, user_id, page, h, v, (error, codePositions) ->
return next(error) if error?
res.send JSON.stringify {
code: codePositions
}
wordcount: (req, res, next = (error) ->) ->
file = req.query.file || "main.tex"
project_id = req.params.project_id
user_id = req.params.user_id
image = req.query.image
logger.log {image, file, project_id}, "word count request"
CompileManager.wordcount project_id, user_id, file, image, (error, result) ->
return next(error) if error?
res.send JSON.stringify {
texcount: result
}
status: (req, res, next = (error)-> )->
res.send("OK")

View File

@@ -1,84 +1,253 @@
ResourceWriter = require "./ResourceWriter"
LatexRunner = require "./LatexRunner"
OutputFileFinder = require "./OutputFileFinder"
OutputCacheManager = require "./OutputCacheManager"
Settings = require("settings-sharelatex")
Path = require "path"
logger = require "logger-sharelatex"
Metrics = require "./Metrics"
child_process = require "child_process"
DraftModeManager = require "./DraftModeManager"
TikzManager = require "./TikzManager"
LockManager = require "./LockManager"
fs = require("fs")
fse = require "fs-extra"
os = require("os")
async = require "async"
Errors = require './Errors'
commandRunner = Settings.clsi?.commandRunner or "./CommandRunner"
logger.info commandRunner:commandRunner, "selecting command runner for clsi"
CommandRunner = require(commandRunner)
getCompileName = (project_id, user_id) ->
if user_id? then "#{project_id}-#{user_id}" else project_id
getCompileDir = (project_id, user_id) ->
Path.join(Settings.path.compilesDir, getCompileName(project_id, user_id))
module.exports = CompileManager =
doCompileWithLock: (request, callback = (error, outputFiles) ->) ->
compileDir = getCompileDir(request.project_id, request.user_id)
lockFile = Path.join(compileDir, ".project-lock")
# use a .project-lock file in the compile directory to prevent
# simultaneous compiles
fse.ensureDir compileDir, (error) ->
return callback(error) if error?
LockManager.runWithLock lockFile, (releaseLock) ->
CompileManager.doCompile(request, releaseLock)
, callback
doCompile: (request, callback = (error, outputFiles) ->) ->
compileDir = Path.join(Settings.path.compilesDir, request.project_id)
compileDir = getCompileDir(request.project_id, request.user_id)
timer = new Metrics.Timer("write-to-disk")
logger.log project_id: request.project_id, "starting compile"
ResourceWriter.syncResourcesToDisk request.project_id, request.resources, compileDir, (error) ->
return callback(error) if error?
logger.log project_id: request.project_id, time_taken: Date.now() - timer.start, "written files to disk"
logger.log project_id: request.project_id, user_id: request.user_id, "syncing resources to disk"
ResourceWriter.syncResourcesToDisk request, compileDir, (error, resourceList) ->
# NOTE: resourceList is insecure, it should only be used to exclude files from the output list
if error? and error instanceof Errors.FilesOutOfSyncError
logger.warn project_id: request.project_id, user_id: request.user_id, "files out of sync, please retry"
return callback(error)
else if error?
logger.err err:error, project_id: request.project_id, user_id: request.user_id, "error writing resources to disk"
return callback(error)
logger.log project_id: request.project_id, user_id: request.user_id, time_taken: Date.now() - timer.start, "written files to disk"
timer.done()
timer = new Metrics.Timer("run-compile")
Metrics.inc("compiles")
LatexRunner.runLatex request.project_id, {
directory: compileDir
mainFile: request.rootResourcePath
compiler: request.compiler
timeout: request.timeout
}, (error) ->
return callback(error) if error?
logger.log project_id: request.project_id, time_taken: Date.now() - timer.start, "done compile"
timer.done()
injectDraftModeIfRequired = (callback) ->
if request.draft
DraftModeManager.injectDraftMode Path.join(compileDir, request.rootResourcePath), callback
else
callback()
OutputFileFinder.findOutputFiles request.resources, compileDir, (error, outputFiles) ->
createTikzFileIfRequired = (callback) ->
TikzManager.checkMainFile compileDir, request.rootResourcePath, resourceList, (error, usesTikzExternalize) ->
return callback(error) if error?
callback null, outputFiles
if usesTikzExternalize
TikzManager.injectOutputFile compileDir, request.rootResourcePath, callback
else
callback()
# set up environment variables for chktex
env = {}
# only run chktex on LaTeX files (not knitr .Rtex files or any others)
isLaTeXFile = request.rootResourcePath?.match(/\.tex$/i)
if request.check? and isLaTeXFile
env['CHKTEX_OPTIONS'] = '-nall -e9 -e10 -w15 -w16'
env['CHKTEX_ULIMIT_OPTIONS'] = '-t 5 -v 64000'
if request.check is 'error'
env['CHKTEX_EXIT_ON_ERROR'] = 1
if request.check is 'validate'
env['CHKTEX_VALIDATE'] = 1
# apply a series of file modifications/creations for draft mode and tikz
async.series [injectDraftModeIfRequired, createTikzFileIfRequired], (error) ->
return callback(error) if error?
timer = new Metrics.Timer("run-compile")
# find the image tag to log it as a metric, e.g. 2015.1 (convert . to - for graphite)
tag = request.imageName?.match(/:(.*)/)?[1]?.replace(/\./g,'-') or "default"
tag = "other" if not request.project_id.match(/^[0-9a-f]{24}$/) # exclude smoke test
Metrics.inc("compiles")
Metrics.inc("compiles-with-image.#{tag}")
compileName = getCompileName(request.project_id, request.user_id)
LatexRunner.runLatex compileName, {
directory: compileDir
mainFile: request.rootResourcePath
compiler: request.compiler
timeout: request.timeout
image: request.imageName
environment: env
}, (error, output, stats, timings) ->
# request was for validation only
if request.check is "validate"
result = if error?.code then "fail" else "pass"
error = new Error("validation")
error.validate = result
# request was for compile, and failed on validation
if request.check is "error" and error?.message is 'exited'
error = new Error("compilation")
error.validate = "fail"
# compile was killed by user, was a validation, or a compile which failed validation
if error?.terminated or error?.validate
OutputFileFinder.findOutputFiles resourceList, compileDir, (err, outputFiles) ->
return callback(err) if err?
callback(error, outputFiles) # return output files so user can check logs
return
# compile completed normally
return callback(error) if error?
Metrics.inc("compiles-succeeded")
for metric_key, metric_value of stats or {}
Metrics.count(metric_key, metric_value)
for metric_key, metric_value of timings or {}
Metrics.timing(metric_key, metric_value)
loadavg = os.loadavg?()
Metrics.gauge("load-avg", loadavg[0]) if loadavg?
ts = timer.done()
logger.log {project_id: request.project_id, user_id: request.user_id, time_taken: ts, stats:stats, timings:timings, loadavg:loadavg}, "done compile"
if stats?["latex-runs"] > 0
Metrics.timing("run-compile-per-pass", ts / stats["latex-runs"])
if stats?["latex-runs"] > 0 and timings?["cpu-time"] > 0
Metrics.timing("run-compile-cpu-time-per-pass", timings["cpu-time"] / stats["latex-runs"])
OutputFileFinder.findOutputFiles resourceList, compileDir, (error, outputFiles) ->
return callback(error) if error?
OutputCacheManager.saveOutputFiles outputFiles, compileDir, (error, newOutputFiles) ->
callback null, newOutputFiles
clearProject: (project_id, _callback = (error) ->) ->
stopCompile: (project_id, user_id, callback = (error) ->) ->
compileName = getCompileName(project_id, user_id)
LatexRunner.killLatex compileName, callback
clearProject: (project_id, user_id, _callback = (error) ->) ->
callback = (error) ->
_callback(error)
_callback = () ->
compileDir = Path.join(Settings.path.compilesDir, project_id)
proc = child_process.spawn "rm", ["-r", compileDir]
compileDir = getCompileDir(project_id, user_id)
proc.on "error", callback
CompileManager._checkDirectory compileDir, (err, exists) ->
return callback(err) if err?
return callback() if not exists # skip removal if no directory present
stderr = ""
proc.stderr.on "data", (chunk) -> stderr += chunk.toString()
proc = child_process.spawn "rm", ["-r", compileDir]
proc.on "close", (code) ->
if code == 0
return callback(null)
proc.on "error", callback
stderr = ""
proc.stderr.on "data", (chunk) -> stderr += chunk.toString()
proc.on "close", (code) ->
if code == 0
return callback(null)
else
return callback(new Error("rm -r #{compileDir} failed: #{stderr}"))
_findAllDirs: (callback = (error, allDirs) ->) ->
root = Settings.path.compilesDir
fs.readdir root, (err, files) ->
return callback(err) if err?
allDirs = (Path.join(root, file) for file in files)
callback(null, allDirs)
clearExpiredProjects: (max_cache_age_ms, callback = (error) ->) ->
now = Date.now()
# action for each directory
expireIfNeeded = (checkDir, cb) ->
fs.stat checkDir, (err, stats) ->
return cb() if err? # ignore errors checking directory
age = now - stats.mtime
hasExpired = (age > max_cache_age_ms)
if hasExpired then fse.remove(checkDir, cb) else cb()
# iterate over all project directories
CompileManager._findAllDirs (error, allDirs) ->
return callback() if error?
async.eachSeries allDirs, expireIfNeeded, callback
_checkDirectory: (compileDir, callback = (error, exists) ->) ->
fs.lstat compileDir, (err, stats) ->
if err?.code is 'ENOENT'
return callback(null, false) # directory does not exist
else if err?
logger.err {dir: compileDir, err:err}, "error on stat of project directory for removal"
return callback(err)
else if not stats?.isDirectory()
logger.err {dir: compileDir, stats:stats}, "bad project directory for removal"
return callback new Error("project directory is not directory")
else
return callback(new Error("rm -r #{compileDir} failed: #{stderr}"))
callback(null, true) # directory exists
syncFromCode: (project_id, file_name, line, column, callback = (error, pdfPositions) ->) ->
syncFromCode: (project_id, user_id, file_name, line, column, callback = (error, pdfPositions) ->) ->
# If LaTeX was run in a virtual environment, the file path that synctex expects
# might not match the file path on the host. The .synctex.gz file however, will be accessed
# wherever it is on the host.
base_dir = Settings.path.synctexBaseDir(project_id)
compileName = getCompileName(project_id, user_id)
base_dir = Settings.path.synctexBaseDir(compileName)
file_path = base_dir + "/" + file_name
synctex_path = Path.join(Settings.path.compilesDir, project_id, "output.pdf")
compileDir = getCompileDir(project_id, user_id)
synctex_path = Path.join(compileDir, "output.pdf")
CompileManager._runSynctex ["code", synctex_path, file_path, line, column], (error, stdout) ->
return callback(error) if error?
logger.log project_id: project_id, file_name: file_name, line: line, column: column, stdout: stdout, "synctex code output"
logger.log project_id: project_id, user_id:user_id, file_name: file_name, line: line, column: column, stdout: stdout, "synctex code output"
callback null, CompileManager._parseSynctexFromCodeOutput(stdout)
syncFromPdf: (project_id, page, h, v, callback = (error, filePositions) ->) ->
base_dir = Settings.path.synctexBaseDir(project_id)
synctex_path = Path.join(Settings.path.compilesDir, project_id, "output.pdf")
syncFromPdf: (project_id, user_id, page, h, v, callback = (error, filePositions) ->) ->
compileName = getCompileName(project_id, user_id)
base_dir = Settings.path.synctexBaseDir(compileName)
compileDir = getCompileDir(project_id, user_id)
synctex_path = Path.join(compileDir, "output.pdf")
CompileManager._runSynctex ["pdf", synctex_path, page, h, v], (error, stdout) ->
return callback(error) if error?
logger.log project_id: project_id, page: page, h: h, v:v, stdout: stdout, "synctex pdf output"
logger.log project_id: project_id, user_id:user_id, page: page, h: h, v:v, stdout: stdout, "synctex pdf output"
callback null, CompileManager._parseSynctexFromPdfOutput(stdout, base_dir)
_checkFileExists: (path, callback = (error) ->) ->
synctexDir = Path.dirname(path)
synctexFile = Path.join(synctexDir, "output.synctex.gz")
fs.stat synctexDir, (error, stats) ->
if error?.code is 'ENOENT'
return callback(new Errors.NotFoundError("called synctex with no output directory"))
return callback(error) if error?
fs.stat synctexFile, (error, stats) ->
if error?.code is 'ENOENT'
return callback(new Errors.NotFoundError("called synctex with no output file"))
return callback(error) if error?
return callback(new Error("not a file")) if not stats?.isFile()
callback()
_runSynctex: (args, callback = (error, stdout) ->) ->
bin_path = Path.resolve(__dirname + "/../../bin/synctex")
seconds = 1000
child_process.execFile bin_path, args, timeout: 10 * seconds, (error, stdout, stderr) ->
outputFilePath = args[1]
CompileManager._checkFileExists outputFilePath, (error) ->
return callback(error) if error?
callback(null, stdout)
if Settings.clsi?.synctexCommandWrapper?
[bin_path, args] = Settings.clsi?.synctexCommandWrapper bin_path, args
child_process.execFile bin_path, args, timeout: 10 * seconds, (error, stdout, stderr) ->
if error?
logger.err err:error, args:args, "error running synctex"
return callback(error)
callback(null, stdout)
_parseSynctexFromCodeOutput: (output) ->
results = []
@@ -105,4 +274,59 @@ module.exports = CompileManager =
line: parseInt(line, 10)
column: parseInt(column, 10)
}
return results
return results
wordcount: (project_id, user_id, file_name, image, callback = (error, pdfPositions) ->) ->
logger.log project_id:project_id, user_id:user_id, file_name:file_name, image:image, "running wordcount"
file_path = "$COMPILE_DIR/" + file_name
command = [ "texcount", '-nocol', '-inc', file_path, "-out=" + file_path + ".wc"]
directory = getCompileDir(project_id, user_id)
timeout = 10 * 1000
compileName = getCompileName(project_id, user_id)
CommandRunner.run compileName, command, directory, image, timeout, {}, (error) ->
return callback(error) if error?
fs.readFile directory + "/" + file_name + ".wc", "utf-8", (err, stdout) ->
if err?
logger.err err:err, command:command, directory:directory, project_id:project_id, user_id:user_id, "error reading word count output"
return callback(err)
results = CompileManager._parseWordcountFromOutput(stdout)
logger.log project_id:project_id, user_id:user_id, wordcount: results, "word count results"
callback null, results
_parseWordcountFromOutput: (output) ->
results = {
encode: ""
textWords: 0
headWords: 0
outside: 0
headers: 0
elements: 0
mathInline: 0
mathDisplay: 0
errors: 0
messages: ""
}
for line in output.split("\n")
[data, info] = line.split(":")
if data.indexOf("Encoding") > -1
results['encode'] = info.trim()
if data.indexOf("in text") > -1
results['textWords'] = parseInt(info, 10)
if data.indexOf("in head") > -1
results['headWords'] = parseInt(info, 10)
if data.indexOf("outside") > -1
results['outside'] = parseInt(info, 10)
if data.indexOf("of head") > -1
results['headers'] = parseInt(info, 10)
if data.indexOf("Number of floats/tables/figures") > -1
results['elements'] = parseInt(info, 10)
if data.indexOf("Number of math inlines") > -1
results['mathInline'] = parseInt(info, 10)
if data.indexOf("Number of math displayed") > -1
results['mathDisplay'] = parseInt(info, 10)
if data is "(errors" # errors reported as (errors:123)
results['errors'] = parseInt(info, 10)
if line.indexOf("!!! ") > -1 # errors logged as !!! message !!!
results['messages'] += line + "\n"
return results

View File

@@ -0,0 +1,24 @@
Path = require 'path'
# here we coerce html, css and js to text/plain,
# otherwise choose correct mime type based on file extension,
# falling back to octet-stream
module.exports = ContentTypeMapper =
map: (path) ->
switch Path.extname(path)
when '.txt', '.html', '.js', '.css', '.svg'
return 'text/plain'
when '.csv'
return 'text/csv'
when '.pdf'
return 'application/pdf'
when '.png'
return 'image/png'
when '.jpg', '.jpeg'
return 'image/jpeg'
when '.tiff'
return 'image/tiff'
when '.gif'
return 'image/gif'
else
return 'application/octet-stream'

View File

@@ -0,0 +1,24 @@
fs = require "fs"
logger = require "logger-sharelatex"
module.exports = DraftModeManager =
injectDraftMode: (filename, callback = (error) ->) ->
fs.readFile filename, "utf8", (error, content) ->
return callback(error) if error?
# avoid adding draft mode more than once
if content?.indexOf("\\documentclass\[draft") >= 0
return callback()
modified_content = DraftModeManager._injectDraftOption content
logger.log {
content: content.slice(0,1024), # \documentclass is normally v near the top
modified_content: modified_content.slice(0,1024),
filename
}, "injected draft class"
fs.writeFile filename, modified_content, callback
_injectDraftOption: (content) ->
content
# With existing options (must be first, otherwise both are applied)
.replace(/\\documentclass\[/g, "\\documentclass[draft,")
# Without existing options
.replace(/\\documentclass\{/g, "\\documentclass[draft]{")

25
app/coffee/Errors.coffee Normal file
View File

@@ -0,0 +1,25 @@
NotFoundError = (message) ->
error = new Error(message)
error.name = "NotFoundError"
error.__proto__ = NotFoundError.prototype
return error
NotFoundError.prototype.__proto__ = Error.prototype
FilesOutOfSyncError = (message) ->
error = new Error(message)
error.name = "FilesOutOfSyncError"
error.__proto__ = FilesOutOfSyncError.prototype
return error
FilesOutOfSyncError.prototype.__proto__ = Error.prototype
AlreadyCompilingError = (message) ->
error = new Error(message)
error.name = "AlreadyCompilingError"
error.__proto__ = AlreadyCompilingError.prototype
return error
AlreadyCompilingError.prototype.__proto__ = Error.prototype
module.exports = Errors =
NotFoundError: NotFoundError
FilesOutOfSyncError: FilesOutOfSyncError
AlreadyCompilingError: AlreadyCompilingError

View File

@@ -4,17 +4,19 @@ logger = require "logger-sharelatex"
Metrics = require "./Metrics"
CommandRunner = require(Settings.clsi?.commandRunner or "./CommandRunner")
ProcessTable = {} # table of currently running jobs (pids or docker container names)
module.exports = LatexRunner =
runLatex: (project_id, options, callback = (error) ->) ->
{directory, mainFile, compiler, timeout} = options
{directory, mainFile, compiler, timeout, image, environment} = options
compiler ||= "pdflatex"
timeout ||= 60000 # milliseconds
logger.log directory: directory, compiler: compiler, timeout: timeout, mainFile: mainFile, "starting compile"
logger.log directory: directory, compiler: compiler, timeout: timeout, mainFile: mainFile, environment: environment, "starting compile"
# We want to run latexmk on the tex file which we will automatically
# generate from the Rtex/Rmd/md file.
mainFile = mainFile.replace(/\.(Rtex|md|Rmd)$/, ".tex")
mainFile = mainFile.replace(/\.(Rtex|md|Rmd)$/, ".md")
if compiler == "pdflatex"
command = LatexRunner._pdflatexCommand mainFile
@@ -26,32 +28,73 @@ module.exports = LatexRunner =
command = LatexRunner._lualatexCommand mainFile
else
return callback new Error("unknown compiler: #{compiler}")
if Settings.clsi?.strace
command = ["strace", "-o", "strace", "-ff"].concat(command)
CommandRunner.run project_id, command, directory, timeout, callback
# ignore the above and make a pandoc command
console.log(mainFile)
console.log(image)
image = "ivotron/pandoc"
command = ["-o", "$COMPILE_DIR/output.html", "/compile/" + mainFile]
_latexmkBaseCommand: [ "latexmk", "-cd", "-f", "-jobname=output", "-auxdir=$COMPILE_DIR", "-outdir=$COMPILE_DIR"]
id = "#{project_id}" # record running project under this id
ProcessTable[id] = CommandRunner.run project_id, command, directory, image, timeout, environment, (error, output) ->
delete ProcessTable[id]
return callback(error) if error?
runs = output?.stderr?.match(/^Run number \d+ of .*latex/mg)?.length or 0
failed = if output?.stdout?.match(/^Latexmk: Errors/m)? then 1 else 0
# counters from latexmk output
stats = {}
stats["latexmk-errors"] = failed
stats["latex-runs"] = runs
stats["latex-runs-with-errors"] = if failed then runs else 0
stats["latex-runs-#{runs}"] = 1
stats["latex-runs-with-errors-#{runs}"] = if failed then 1 else 0
# timing information from /usr/bin/time
timings = {}
stderr = output?.stderr
timings["cpu-percent"] = stderr?.match(/Percent of CPU this job got: (\d+)/m)?[1] or 0
timings["cpu-time"] = stderr?.match(/User time.*: (\d+.\d+)/m)?[1] or 0
timings["sys-time"] = stderr?.match(/System time.*: (\d+.\d+)/m)?[1] or 0
callback error, output, stats, timings
killLatex: (project_id, callback = (error) ->) ->
id = "#{project_id}"
logger.log {id:id}, "killing running compile"
if not ProcessTable[id]?
logger.warn {id}, "no such project to kill"
return callback(null)
else
CommandRunner.kill ProcessTable[id], callback
_latexmkBaseCommand: (Settings?.clsi?.latexmkCommandPrefix || []).concat([
"latexmk", "-cd", "-f", "-jobname=output", "-auxdir=$COMPILE_DIR", "-outdir=$COMPILE_DIR",
"-synctex=1","-interaction=batchmode"
])
_pdflatexCommand: (mainFile) ->
LatexRunner._latexmkBaseCommand.concat [
"-pdf", "-e", "$pdflatex='pdflatex -synctex=1 -interaction=batchmode %O %S'",
"-pdf",
Path.join("$COMPILE_DIR", mainFile)
]
_latexCommand: (mainFile) ->
LatexRunner._latexmkBaseCommand.concat [
"-pdfdvi", "-e", "$latex='latex -synctex=1 -interaction=batchmode %O %S'",
"-pdfdvi",
Path.join("$COMPILE_DIR", mainFile)
]
_xelatexCommand: (mainFile) ->
LatexRunner._latexmkBaseCommand.concat [
"-xelatex", "-e", "$pdflatex='xelatex -synctex=1 -interaction=batchmode %O %S'",
"-xelatex",
Path.join("$COMPILE_DIR", mainFile)
]
_lualatexCommand: (mainFile) ->
LatexRunner._latexmkBaseCommand.concat [
"-pdf", "-e", "$pdflatex='lualatex -synctex=1 -interaction=batchmode %O %S'",
"-lualatex",
Path.join("$COMPILE_DIR", mainFile)
]

View File

@@ -0,0 +1,23 @@
Settings = require('settings-sharelatex')
logger = require "logger-sharelatex"
Lockfile = require('lockfile') # from https://github.com/npm/lockfile
Errors = require "./Errors"
module.exports = LockManager =
LOCK_TEST_INTERVAL: 1000 # 50ms between each test of the lock
MAX_LOCK_WAIT_TIME: 15000 # 10s maximum time to spend trying to get the lock
LOCK_STALE: 5*60*1000 # 5 mins time until lock auto expires
runWithLock: (path, runner = ((releaseLock = (error) ->) ->), callback = ((error) ->)) ->
lockOpts =
wait: @MAX_LOCK_WAIT_TIME
pollPeriod: @LOCK_TEST_INTERVAL
stale: @LOCK_STALE
Lockfile.lock path, lockOpts, (error) ->
return callback new Errors.AlreadyCompilingError("compile in progress") if error?.code is 'EEXIST'
return callback(error) if error?
runner (error1, args...) ->
Lockfile.unlock path, (error2) ->
error = error1 or error2
return callback(error) if error?
callback(null, args...)

View File

@@ -0,0 +1,199 @@
async = require "async"
fs = require "fs"
fse = require "fs-extra"
Path = require "path"
logger = require "logger-sharelatex"
_ = require "underscore"
Settings = require "settings-sharelatex"
crypto = require "crypto"
OutputFileOptimiser = require "./OutputFileOptimiser"
module.exports = OutputCacheManager =
CACHE_SUBDIR: '.cache/clsi'
ARCHIVE_SUBDIR: '.archive/clsi'
# build id is HEXDATE-HEXRANDOM from Date.now()and RandomBytes
# for backwards compatibility, make the randombytes part optional
BUILD_REGEX: /^[0-9a-f]+(-[0-9a-f]+)?$/
CACHE_LIMIT: 2 # maximum number of cache directories
CACHE_AGE: 60*60*1000 # up to one hour old
path: (buildId, file) ->
# used by static server, given build id return '.cache/clsi/buildId'
if buildId.match OutputCacheManager.BUILD_REGEX
return Path.join(OutputCacheManager.CACHE_SUBDIR, buildId, file)
else
# for invalid build id, return top level
return file
generateBuildId: (callback = (error, buildId) ->) ->
# generate a secure build id from Date.now() and 8 random bytes in hex
crypto.randomBytes 8, (err, buf) ->
return callback(err) if err?
random = buf.toString('hex')
date = Date.now().toString(16)
callback err, "#{date}-#{random}"
saveOutputFiles: (outputFiles, compileDir, callback = (error) ->) ->
OutputCacheManager.generateBuildId (err, buildId) ->
return callback(err) if err?
OutputCacheManager.saveOutputFilesInBuildDir outputFiles, compileDir, buildId, callback
saveOutputFilesInBuildDir: (outputFiles, compileDir, buildId, callback = (error) ->) ->
# make a compileDir/CACHE_SUBDIR/build_id directory and
# copy all the output files into it
cacheRoot = Path.join(compileDir, OutputCacheManager.CACHE_SUBDIR)
# Put the files into a new cache subdirectory
cacheDir = Path.join(compileDir, OutputCacheManager.CACHE_SUBDIR, buildId)
# Is it a per-user compile? check if compile directory is PROJECTID-USERID
perUser = Path.basename(compileDir).match(/^[0-9a-f]{24}-[0-9a-f]{24}$/)
# Archive logs in background
if Settings.clsi?.archive_logs or Settings.clsi?.strace
OutputCacheManager.archiveLogs outputFiles, compileDir, buildId, (err) ->
if err?
logger.warn err:err, "erroring archiving log files"
# make the new cache directory
fse.ensureDir cacheDir, (err) ->
if err?
logger.error err: err, directory: cacheDir, "error creating cache directory"
callback(err, outputFiles)
else
# copy all the output files into the new cache directory
results = []
async.mapSeries outputFiles, (file, cb) ->
# don't send dot files as output, express doesn't serve them
if OutputCacheManager._fileIsHidden(file.path)
logger.warn compileDir: compileDir, path: file.path, "ignoring dotfile in output"
return cb()
# copy other files into cache directory if valid
newFile = _.clone(file)
[src, dst] = [Path.join(compileDir, file.path), Path.join(cacheDir, file.path)]
OutputCacheManager._checkFileIsSafe src, (err, isSafe) ->
return cb(err) if err?
if !isSafe
return cb()
OutputCacheManager._checkIfShouldCopy src, (err, shouldCopy) ->
return cb(err) if err?
if !shouldCopy
return cb()
OutputCacheManager._copyFile src, dst, (err) ->
return cb(err) if err?
newFile.build = buildId # attach a build id if we cached the file
results.push newFile
cb()
, (err) ->
if err?
# pass back the original files if we encountered *any* error
callback(err, outputFiles)
# clean up the directory we just created
fse.remove cacheDir, (err) ->
if err?
logger.error err: err, dir: cacheDir, "error removing cache dir after failure"
else
# pass back the list of new files in the cache
callback(err, results)
# let file expiry run in the background, expire all previous files if per-user
OutputCacheManager.expireOutputFiles cacheRoot, {keep: buildId, limit: if perUser then 1 else null}
archiveLogs: (outputFiles, compileDir, buildId, callback = (error) ->) ->
archiveDir = Path.join(compileDir, OutputCacheManager.ARCHIVE_SUBDIR, buildId)
logger.log {dir: archiveDir}, "archiving log files for project"
fse.ensureDir archiveDir, (err) ->
return callback(err) if err?
async.mapSeries outputFiles, (file, cb) ->
[src, dst] = [Path.join(compileDir, file.path), Path.join(archiveDir, file.path)]
OutputCacheManager._checkFileIsSafe src, (err, isSafe) ->
return cb(err) if err?
return cb() if !isSafe
OutputCacheManager._checkIfShouldArchive src, (err, shouldArchive) ->
return cb(err) if err?
return cb() if !shouldArchive
OutputCacheManager._copyFile src, dst, cb
, callback
expireOutputFiles: (cacheRoot, options, callback = (error) ->) ->
# look in compileDir for build dirs and delete if > N or age of mod time > T
fs.readdir cacheRoot, (err, results) ->
if err?
return callback(null) if err.code == 'ENOENT' # cache directory is empty
logger.error err: err, project_id: cacheRoot, "error clearing cache"
return callback(err)
dirs = results.sort().reverse()
currentTime = Date.now()
isExpired = (dir, index) ->
return false if options?.keep == dir
# remove any directories over the requested (non-null) limit
return true if options?.limit? and index > options.limit
# remove any directories over the hard limit
return true if index > OutputCacheManager.CACHE_LIMIT
# we can get the build time from the first part of the directory name DDDD-RRRR
# DDDD is date and RRRR is random bytes
dirTime = parseInt(dir.split('-')?[0], 16)
age = currentTime - dirTime
return age > OutputCacheManager.CACHE_AGE
toRemove = _.filter(dirs, isExpired)
removeDir = (dir, cb) ->
fse.remove Path.join(cacheRoot, dir), (err, result) ->
logger.log cache: cacheRoot, dir: dir, "removed expired cache dir"
if err?
logger.error err: err, dir: dir, "cache remove error"
cb(err, result)
async.eachSeries toRemove, (dir, cb) ->
removeDir dir, cb
, callback
_fileIsHidden: (path) ->
return path?.match(/^\.|\/./)?
_checkFileIsSafe: (src, callback = (error, isSafe) ->) ->
# check if we have a valid file to copy into the cache
fs.stat src, (err, stats) ->
if err?.code is 'ENOENT'
logger.warn err: err, file: src, "file has disappeared before copying to build cache"
callback(err, false)
else if err?
# some other problem reading the file
logger.error err: err, file: src, "stat error for file in cache"
callback(err, false)
else if not stats.isFile()
# other filetype - reject it
logger.warn src: src, stat: stats, "nonfile output - refusing to copy to cache"
callback(null, false)
else
# it's a plain file, ok to copy
callback(null, true)
_copyFile: (src, dst, callback) ->
# copy output file into the cache
fse.copy src, dst, (err) ->
if err?.code is 'ENOENT'
logger.warn err: err, file: src, "file has disappeared when copying to build cache"
callback(err, false)
else if err?
logger.error err: err, src: src, dst: dst, "copy error for file in cache"
callback(err)
else
if Settings.clsi?.optimiseInDocker
# don't run any optimisations on the pdf when they are done
# in the docker container
callback()
else
# call the optimiser for the file too
OutputFileOptimiser.optimiseFile src, dst, callback
_checkIfShouldCopy: (src, callback = (err, shouldCopy) ->) ->
return callback(null, !Path.basename(src).match(/^strace/))
_checkIfShouldArchive: (src, callback = (err, shouldCopy) ->) ->
if Path.basename(src).match(/^strace/)
return callback(null, true)
if Settings.clsi?.archive_logs and Path.basename(src) in ["output.log", "output.blg"]
return callback(null, true)
return callback(null, false)

View File

@@ -1,58 +1,52 @@
async = require "async"
fs = require "fs"
Path = require "path"
wrench = require "wrench"
spawn = require("child_process").spawn
logger = require "logger-sharelatex"
module.exports = OutputFileFinder =
findOutputFiles: (resources, directory, callback = (error, outputFiles) ->) ->
findOutputFiles: (resources, directory, callback = (error, outputFiles, allFiles) ->) ->
incomingResources = {}
for resource in resources
incomingResources[resource.path] = true
logger.log directory: directory, "getting output files"
OutputFileFinder._getAllFiles directory, (error, allFiles) ->
jobs = []
OutputFileFinder._getAllFiles directory, (error, allFiles = []) ->
if error?
logger.err err:error, "error finding all output files"
return callback(error)
outputFiles = []
for file in allFiles
do (file) ->
jobs.push (callback) ->
if incomingResources[file.path]
return callback()
else
OutputFileFinder._isDirectory Path.join(directory, file.path), (error, directory) ->
return callback(error) if error?
if !directory
outputFiles.push file
callback()
if !incomingResources[file]
outputFiles.push {
path: file
type: file.match(/\.([^\.]+)$/)?[1]
}
callback null, outputFiles, allFiles
async.series jobs, (error) ->
return callback(error) if error?
callback null, outputFiles
_isDirectory: (path, callback = (error, directory) ->) ->
fs.stat path, (error, stat) ->
callback error, stat?.isDirectory()
_getAllFiles: (directory, _callback = (error, outputFiles) ->) ->
callback = (error, outputFiles) ->
_callback(error, outputFiles)
_getAllFiles: (directory, _callback = (error, fileList) ->) ->
callback = (error, fileList) ->
_callback(error, fileList)
_callback = () ->
outputFiles = []
# don't include clsi-specific files/directories in the output list
EXCLUDE_DIRS = ["-name", ".cache", "-o", "-name", ".archive","-o", "-name", ".project-*"]
args = [directory, "(", EXCLUDE_DIRS..., ")", "-prune", "-o", "-type", "f", "-print"]
logger.log args: args, "running find command"
wrench.readdirRecursive directory, (error, files) =>
if error?
if error.code == "ENOENT"
# Directory doesn't exist, which is not a problem
return callback(null, [])
else
return callback(error)
# readdirRecursive returns multiple times and finishes with a null response
if !files?
return callback(null, outputFiles)
for file in files
outputFiles.push
path: file
type: file.match(/\.([^\.]+)$/)?[1]
proc = spawn("find", args)
stdout = ""
proc.stdout.on "data", (chunk) ->
stdout += chunk.toString()
proc.on "error", callback
proc.on "close", (code) ->
if code != 0
logger.warn {directory, code}, "find returned error, directory likely doesn't exist"
return callback null, []
fileList = stdout.trim().split("\n")
fileList = fileList.map (file) ->
# Strip leading directory
path = Path.relative(directory, file)
return callback null, fileList

View File

@@ -0,0 +1,55 @@
fs = require "fs"
Path = require "path"
spawn = require("child_process").spawn
logger = require "logger-sharelatex"
Metrics = require "./Metrics"
_ = require "underscore"
module.exports = OutputFileOptimiser =
optimiseFile: (src, dst, callback = (error) ->) ->
# check output file (src) and see if we can optimise it, storing
# the result in the build directory (dst)
if src.match(/\/output\.pdf$/)
OutputFileOptimiser.checkIfPDFIsOptimised src, (err, isOptimised) ->
return callback(null) if err? or isOptimised
OutputFileOptimiser.optimisePDF src, dst, callback
else
callback (null)
checkIfPDFIsOptimised: (file, callback) ->
SIZE = 16*1024 # check the header of the pdf
result = new Buffer(SIZE)
result.fill(0) # prevent leakage of uninitialised buffer
fs.open file, "r", (err, fd) ->
return callback(err) if err?
fs.read fd, result, 0, SIZE, 0, (errRead, bytesRead, buffer) ->
fs.close fd, (errClose) ->
return callback(errRead) if errRead?
return callback(errClose) if errReadClose?
isOptimised = buffer.toString('ascii').indexOf("/Linearized 1") >= 0
callback(null, isOptimised)
optimisePDF: (src, dst, callback = (error) ->) ->
tmpOutput = dst + '.opt'
args = ["--linearize", src, tmpOutput]
logger.log args: args, "running qpdf command"
timer = new Metrics.Timer("qpdf")
proc = spawn("qpdf", args)
stdout = ""
proc.stdout.on "data", (chunk) ->
stdout += chunk.toString()
callback = _.once(callback) # avoid double call back for error and close event
proc.on "error", (err) ->
logger.warn {err, args}, "qpdf failed"
callback(null) # ignore the error
proc.on "close", (code) ->
timer.done()
if code != 0
logger.warn {code, args}, "qpdf returned error"
return callback(null) # ignore the error
fs.rename tmpOutput, dst, (err) ->
if err?
logger.warn {tmpOutput, dst}, "failed to rename output of qpdf command"
callback(null) # ignore the error

View File

@@ -3,16 +3,19 @@ CompileManager = require "./CompileManager"
db = require "./db"
async = require "async"
logger = require "logger-sharelatex"
oneDay = 24 * 60 * 60 * 1000
Settings = require "settings-sharelatex"
module.exports = ProjectPersistenceManager =
EXPIRY_TIMEOUT: oneDay = 24 * 60 * 60 * 1000 #ms
EXPIRY_TIMEOUT: Settings.project_cache_length_ms || oneDay * 2.5
markProjectAsJustAccessed: (project_id, callback = (error) ->) ->
db.Project.findOrCreate(project_id: project_id)
.success(
(project) ->
db.Project.findOrCreate(where: {project_id: project_id})
.spread(
(project, created) ->
project.updateAttributes(lastAccessed: new Date())
.success(() -> callback())
.then(() -> callback())
.error callback
)
.error callback
@@ -24,31 +27,40 @@ module.exports = ProjectPersistenceManager =
jobs = for project_id in (project_ids or [])
do (project_id) ->
(callback) ->
ProjectPersistenceManager.clearProject project_id, (err) ->
ProjectPersistenceManager.clearProjectFromCache project_id, (err) ->
if err?
logger.error err: err, project_id: project_id, "error clearing project"
callback()
async.series jobs, callback
clearProject: (project_id, callback = (error) ->) ->
logger.log project_id: project_id, "clearing project"
CompileManager.clearProject project_id, (error) ->
return callback(error) if error?
UrlCache.clearProject project_id, (error) ->
async.series jobs, (error) ->
return callback(error) if error?
ProjectPersistenceManager._clearProjectFromDatabase project_id, (error) ->
return callback(error) if error?
callback()
CompileManager.clearExpiredProjects ProjectPersistenceManager.EXPIRY_TIMEOUT, (error) ->
callback() # ignore any errors from deleting directories
clearProject: (project_id, user_id, callback = (error) ->) ->
logger.log project_id: project_id, user_id:user_id, "clearing project for user"
CompileManager.clearProject project_id, user_id, (error) ->
return callback(error) if error?
ProjectPersistenceManager.clearProjectFromCache project_id, (error) ->
return callback(error) if error?
callback()
clearProjectFromCache: (project_id, callback = (error) ->) ->
logger.log project_id: project_id, "clearing project from cache"
UrlCache.clearProject project_id, (error) ->
return callback(error) if error?
ProjectPersistenceManager._clearProjectFromDatabase project_id, (error) ->
return callback(error) if error?
callback()
_clearProjectFromDatabase: (project_id, callback = (error) ->) ->
db.Project.destroy(project_id: project_id)
.success(() -> callback())
db.Project.destroy(where: {project_id: project_id})
.then(() -> callback())
.error callback
_findExpiredProjectIds: (callback = (error, project_ids) ->) ->
db.Project.findAll(where: ["lastAccessed < ?", new Date(Date.now() - ProjectPersistenceManager.EXPIRY_TIMEOUT)])
.success(
(projects) ->
callback null, projects.map((project) -> project.project_id)
)
.error callback
.then((projects) ->
callback null, projects.map((project) -> project.project_id)
).error callback
logger.log {EXPIRY_TIMEOUT: ProjectPersistenceManager.EXPIRY_TIMEOUT}, "project assets kept timeout"

View File

@@ -1,6 +1,6 @@
module.exports = RequestParser =
VALID_COMPILERS: ["pdflatex", "latex", "xelatex", "lualatex"]
MAX_TIMEOUT: 60
MAX_TIMEOUT: 300
parse: (body, callback = (error, data) ->) ->
response = {}
@@ -21,16 +21,55 @@ module.exports = RequestParser =
compile.options.timeout
default: RequestParser.MAX_TIMEOUT
type: "number"
response.imageName = @_parseAttribute "imageName",
compile.options.imageName,
type: "string"
response.draft = @_parseAttribute "draft",
compile.options.draft,
default: false,
type: "boolean"
response.check = @_parseAttribute "check",
compile.options.check,
type: "string"
# The syncType specifies whether the request contains all
# resources (full) or only those resources to be updated
# in-place (incremental).
response.syncType = @_parseAttribute "syncType",
compile.options.syncType,
validValues: ["full", "incremental"]
type: "string"
# The syncState is an identifier passed in with the request
# which has the property that it changes when any resource is
# added, deleted, moved or renamed.
#
# on syncType full the syncState identifier is passed in and
# stored
#
# on syncType incremental the syncState identifier must match
# the stored value
response.syncState = @_parseAttribute "syncState",
compile.options.syncState,
type: "string"
if response.timeout > RequestParser.MAX_TIMEOUT
response.timeout = RequestParser.MAX_TIMEOUT
response.timeout = response.timeout * 1000 # milliseconds
response.resources = (@_parseResource(resource) for resource in (compile.resources or []))
response.rootResourcePath = @_parseAttribute "rootResourcePath",
rootResourcePath = @_parseAttribute "rootResourcePath",
compile.rootResourcePath
default: "main.tex"
type: "string"
originalRootResourcePath = rootResourcePath
sanitizedRootResourcePath = RequestParser._sanitizePath(rootResourcePath)
response.rootResourcePath = RequestParser._checkPath(sanitizedRootResourcePath)
for resource in response.resources
if resource.path == originalRootResourcePath
resource.path = sanitizedRootResourcePath
catch error
return callback error
@@ -69,6 +108,15 @@ module.exports = RequestParser =
throw "#{name} attribute should be a #{options.type}"
else
return options.default if options.default?
throw "Default not implemented"
return attribute
_sanitizePath: (path) ->
# See http://php.net/manual/en/function.escapeshellcmd.php
path.replace(/[\#\&\;\`\|\*\?\~\<\>\^\(\)\[\]\{\}\$\\\x0A\xFF\x00]/g, "")
_checkPath: (path) ->
# check that the request does not use a relative path
for dir in path.split('/')
if dir == '..'
throw "relative path in root resource"
return path

View File

@@ -0,0 +1,72 @@
Path = require "path"
fs = require "fs"
logger = require "logger-sharelatex"
settings = require("settings-sharelatex")
Errors = require "./Errors"
SafeReader = require "./SafeReader"
module.exports = ResourceStateManager =
# The sync state is an identifier which must match for an
# incremental update to be allowed.
#
# The initial value is passed in and stored on a full
# compile, along with the list of resources..
#
# Subsequent incremental compiles must come with the same value - if
# not they will be rejected with a 409 Conflict response. The
# previous list of resources is returned.
#
# An incremental compile can only update existing files with new
# content. The sync state identifier must change if any docs or
# files are moved, added, deleted or renamed.
SYNC_STATE_FILE: ".project-sync-state"
SYNC_STATE_MAX_SIZE: 128*1024
saveProjectState: (state, resources, basePath, callback = (error) ->) ->
stateFile = Path.join(basePath, @SYNC_STATE_FILE)
if not state? # remove the file if no state passed in
logger.log state:state, basePath:basePath, "clearing sync state"
fs.unlink stateFile, (err) ->
if err? and err.code isnt 'ENOENT'
return callback(err)
else
return callback()
else
logger.log state:state, basePath:basePath, "writing sync state"
resourceList = (resource.path for resource in resources)
fs.writeFile stateFile, [resourceList..., "stateHash:#{state}"].join("\n"), callback
checkProjectStateMatches: (state, basePath, callback = (error, resources) ->) ->
stateFile = Path.join(basePath, @SYNC_STATE_FILE)
size = @SYNC_STATE_MAX_SIZE
SafeReader.readFile stateFile, size, 'utf8', (err, result, bytesRead) ->
return callback(err) if err?
if bytesRead is size
logger.error file:stateFile, size:size, bytesRead:bytesRead, "project state file truncated"
[resourceList..., oldState] = result?.toString()?.split("\n") or []
newState = "stateHash:#{state}"
logger.log state:state, oldState: oldState, basePath:basePath, stateMatches: (newState is oldState), "checking sync state"
if newState isnt oldState
return callback new Errors.FilesOutOfSyncError("invalid state for incremental update")
else
resources = ({path: path} for path in resourceList)
callback(null, resources)
checkResourceFiles: (resources, allFiles, basePath, callback = (error) ->) ->
# check the paths are all relative to current directory
for file in resources or []
for dir in file?.path?.split('/')
if dir == '..'
return callback new Error("relative path in resource file list")
# check if any of the input files are not present in list of files
seenFile = {}
for file in allFiles
seenFile[file] = true
missingFiles = (resource.path for resource in resources when not seenFile[resource.path])
if missingFiles?.length > 0
logger.err missingFiles:missingFiles, basePath:basePath, allFiles:allFiles, resources:resources, "missing input files for project"
return callback new Errors.FilesOutOfSyncError("resource files missing in incremental update")
else
callback()

View File

@@ -4,24 +4,71 @@ fs = require "fs"
async = require "async"
mkdirp = require "mkdirp"
OutputFileFinder = require "./OutputFileFinder"
ResourceStateManager = require "./ResourceStateManager"
Metrics = require "./Metrics"
logger = require "logger-sharelatex"
settings = require("settings-sharelatex")
parallelFileDownloads = settings.parallelFileDownloads or 1
module.exports = ResourceWriter =
syncResourcesToDisk: (project_id, resources, basePath, callback = (error) ->) ->
@_removeExtraneousFiles resources, basePath, (error) =>
syncResourcesToDisk: (request, basePath, callback = (error, resourceList) ->) ->
if request.syncType is "incremental"
logger.log project_id: request.project_id, user_id: request.user_id, "incremental sync"
ResourceStateManager.checkProjectStateMatches request.syncState, basePath, (error, resourceList) ->
return callback(error) if error?
ResourceWriter._removeExtraneousFiles resourceList, basePath, (error, outputFiles, allFiles) ->
return callback(error) if error?
ResourceStateManager.checkResourceFiles resourceList, allFiles, basePath, (error) ->
return callback(error) if error?
ResourceWriter.saveIncrementalResourcesToDisk request.project_id, request.resources, basePath, (error) ->
return callback(error) if error?
callback(null, resourceList)
else
logger.log project_id: request.project_id, user_id: request.user_id, "full sync"
@saveAllResourcesToDisk request.project_id, request.resources, basePath, (error) ->
return callback(error) if error?
ResourceStateManager.saveProjectState request.syncState, request.resources, basePath, (error) ->
return callback(error) if error?
callback(null, request.resources)
saveIncrementalResourcesToDisk: (project_id, resources, basePath, callback = (error) ->) ->
@_createDirectory basePath, (error) =>
return callback(error) if error?
jobs = for resource in resources
do (resource) =>
(callback) => @_writeResourceToDisk(project_id, resource, basePath, callback)
async.series jobs, callback
async.parallelLimit jobs, parallelFileDownloads, callback
_removeExtraneousFiles: (resources, basePath, _callback = (error) ->) ->
saveAllResourcesToDisk: (project_id, resources, basePath, callback = (error) ->) ->
@_createDirectory basePath, (error) =>
return callback(error) if error?
@_removeExtraneousFiles resources, basePath, (error) =>
return callback(error) if error?
jobs = for resource in resources
do (resource) =>
(callback) => @_writeResourceToDisk(project_id, resource, basePath, callback)
async.parallelLimit jobs, parallelFileDownloads, callback
_createDirectory: (basePath, callback = (error) ->) ->
fs.mkdir basePath, (err) ->
if err?
if err.code is 'EEXIST'
return callback()
else
logger.log {err: err, dir:basePath}, "error creating directory"
return callback(err)
else
return callback()
_removeExtraneousFiles: (resources, basePath, _callback = (error, outputFiles, allFiles) ->) ->
timer = new Metrics.Timer("unlink-output-files")
callback = (error) ->
callback = (error, result...) ->
timer.done()
_callback(error)
_callback(error, result...)
OutputFileFinder.findOutputFiles resources, basePath, (error, outputFiles) ->
OutputFileFinder.findOutputFiles resources, basePath, (error, outputFiles, allFiles) ->
return callback(error) if error?
jobs = []
@@ -29,40 +76,55 @@ module.exports = ResourceWriter =
do (file) ->
path = file.path
should_delete = true
if path.match(/^output\./) or path.match(/\.aux$/)
if path.match(/^output\./) or path.match(/\.aux$/) or path.match(/^cache\//) # knitr cache
should_delete = false
if path == "output.pdf" or path == "output.dvi" or path == "output.log"
if path.match(/^output-.*/) # Tikz cached figures
should_delete = false
if path == "output.pdf" or path == "output.dvi" or path == "output.log" or path == "output.xdv"
should_delete = true
if path == "output.tex" # created by TikzManager if present in output files
should_delete = true
if should_delete
jobs.push (callback) -> ResourceWriter._deleteFileIfNotDirectory Path.join(basePath, path), callback
async.series jobs, callback
async.series jobs, (error) ->
return callback(error) if error?
callback(null, outputFiles, allFiles)
_deleteFileIfNotDirectory: (path, callback = (error) ->) ->
fs.stat path, (error, stat) ->
return callback(error) if error?
if stat.isFile()
fs.unlink path, callback
if error? and error.code is 'ENOENT'
return callback()
else if error?
logger.err {err: error, path: path}, "error stating file in deleteFileIfNotDirectory"
return callback(error)
else if stat.isFile()
fs.unlink path, (error) ->
if error?
logger.err {err: error, path: path}, "error removing file in deleteFileIfNotDirectory"
callback(error)
else
callback()
else
callback()
_writeResourceToDisk: (project_id, resource, basePath, callback = (error) ->) ->
path = Path.normalize(Path.join(basePath, resource.path))
if (path.slice(0, basePath.length) != basePath)
return callback new Error("resource path is outside root directory")
mkdirp Path.dirname(path), (error) ->
ResourceWriter.checkPath basePath, resource.path, (error, path) ->
return callback(error) if error?
# TODO: Don't overwrite file if it hasn't been modified
if resource.url?
UrlCache.downloadUrlToFile(
project_id,
resource.url,
path,
resource.modified,
callback
)
else
fs.writeFile path, resource.content, callback
mkdirp Path.dirname(path), (error) ->
return callback(error) if error?
# TODO: Don't overwrite file if it hasn't been modified
if resource.url?
UrlCache.downloadUrlToFile project_id, resource.url, path, resource.modified, (err)->
if err?
logger.err err:err, project_id:project_id, path:path, resource_url:resource.url, modified:resource.modified, "error downloading file for resources"
callback() #try and continue compiling even if http resource can not be downloaded at this time
else
fs.writeFile path, resource.content, callback
checkPath: (basePath, resourcePath, callback) ->
path = Path.normalize(Path.join(basePath, resourcePath))
if (path.slice(0, basePath.length + 1) != basePath + "/")
return callback new Error("resource path is outside root directory")
else
return callback(null, path)

View File

@@ -0,0 +1,25 @@
fs = require "fs"
logger = require "logger-sharelatex"
module.exports = SafeReader =
# safely read up to size bytes from a file and return result as a
# string
readFile: (file, size, encoding, callback = (error, result) ->) ->
fs.open file, 'r', (err, fd) ->
return callback() if err? and err.code is 'ENOENT'
return callback(err) if err?
# safely return always closing the file
callbackWithClose = (err, result...) ->
fs.close fd, (err1) ->
return callback(err) if err?
return callback(err1) if err1?
callback(null, result...)
buff = new Buffer(size, 0) # fill with zeros
fs.read fd, buff, 0, buff.length, 0, (err, bytesRead, buffer) ->
return callbackWithClose(err) if err?
result = buffer.toString(encoding, 0, bytesRead)
callbackWithClose(null, result, bytesRead)

View File

@@ -0,0 +1,41 @@
Path = require("path")
fs = require("fs")
Settings = require("settings-sharelatex")
logger = require("logger-sharelatex")
url = require "url"
module.exports = ForbidSymlinks = (staticFn, root, options) ->
expressStatic = staticFn root, options
basePath = Path.resolve(root)
return (req, res, next) ->
path = url.parse(req.url)?.pathname
# check that the path is of the form /project_id_or_name/path/to/file.log
if result = path.match(/^\/?([a-zA-Z0-9_-]+)\/(.*)/)
project_id = result[1]
file = result[2]
else
logger.warn path: path, "unrecognized file request"
return res.sendStatus(404)
# check that the file does not use a relative path
for dir in file.split('/')
if dir == '..'
logger.warn path: path, "attempt to use a relative path"
return res.sendStatus(404)
# check that the requested path is normalized
requestedFsPath = "#{basePath}/#{project_id}/#{file}"
if requestedFsPath != Path.normalize(requestedFsPath)
logger.error path: requestedFsPath, "requestedFsPath is not normalized"
return res.sendStatus(404)
# check that the requested path is not a symlink
fs.realpath requestedFsPath, (err, realFsPath)->
if err?
if err.code == 'ENOENT'
return res.sendStatus(404)
else
logger.error err:err, requestedFsPath:requestedFsPath, realFsPath:realFsPath, path: req.params[0], project_id: req.params.project_id, "error checking file access"
return res.sendStatus(500)
else if requestedFsPath != realFsPath
logger.warn requestedFsPath:requestedFsPath, realFsPath:realFsPath, path: req.params[0], project_id: req.params.project_id, "trying to access a different file (symlink), aborting"
return res.sendStatus(404)
else
expressStatic(req, res, next)

View File

@@ -0,0 +1,35 @@
fs = require "fs"
Path = require "path"
ResourceWriter = require "./ResourceWriter"
SafeReader = require "./SafeReader"
logger = require "logger-sharelatex"
# for \tikzexternalize to work the main file needs to match the
# jobname. Since we set the -jobname to output, we have to create a
# copy of the main file as 'output.tex'.
module.exports = TikzManager =
checkMainFile: (compileDir, mainFile, resources, callback = (error, usesTikzExternalize) ->) ->
# if there's already an output.tex file, we don't want to touch it
for resource in resources
if resource.path is "output.tex"
logger.log compileDir: compileDir, mainFile: mainFile, "output.tex already in resources"
return callback(null, false)
# if there's no output.tex, see if we are using tikz/pgf in the main file
ResourceWriter.checkPath compileDir, mainFile, (error, path) ->
return callback(error) if error?
SafeReader.readFile path, 65536, "utf8", (error, content) ->
return callback(error) if error?
usesTikzExternalize = content?.indexOf("\\tikzexternalize") >= 0
logger.log compileDir: compileDir, mainFile: mainFile, usesTikzExternalize:usesTikzExternalize, "checked for tikzexternalize"
callback null, usesTikzExternalize
injectOutputFile: (compileDir, mainFile, callback = (error) ->) ->
ResourceWriter.checkPath compileDir, mainFile, (error, path) ->
return callback(error) if error?
fs.readFile path, "utf8", (error, content) ->
return callback(error) if error?
logger.log compileDir: compileDir, mainFile: mainFile, "copied file to output.tex for tikz"
# use wx flag to ensure that output file does not already exist
fs.writeFile Path.join(compileDir, "output.tex"), content, {flag:'wx'}, callback

View File

@@ -10,7 +10,12 @@ module.exports = UrlCache =
downloadUrlToFile: (project_id, url, destPath, lastModified, callback = (error) ->) ->
UrlCache._ensureUrlIsInCache project_id, url, lastModified, (error, pathToCachedUrl) =>
return callback(error) if error?
UrlCache._copyFile(pathToCachedUrl, destPath, callback)
UrlCache._copyFile pathToCachedUrl, destPath, (error) ->
if error?
UrlCache._clearUrlDetails project_id, url, () ->
callback(error)
else
callback(error)
clearProject: (project_id, callback = (error) ->) ->
UrlCache._findAllUrlsInProject project_id, (error, urls) ->
@@ -62,14 +67,17 @@ module.exports = UrlCache =
_copyFile: (from, to, _callback = (error) ->) ->
callbackOnce = (error) ->
if error?
logger.error err: error, from:from, to:to, "error copying file from cache"
_callback(error)
_callback = () ->
writeStream = fs.createWriteStream(to)
readStream = fs.createReadStream(from)
writeStream.on "error", callbackOnce
readStream.on "error", callbackOnce
writeStream.on "close", () -> callbackOnce()
readStream.pipe(writeStream)
writeStream.on "close", callbackOnce
writeStream.on "open", () ->
readStream.pipe(writeStream)
_clearUrlFromCache: (project_id, url, callback = (error) ->) ->
UrlCache._clearUrlDetails project_id, url, (error) ->
@@ -79,31 +87,35 @@ module.exports = UrlCache =
callback null
_deleteUrlCacheFromDisk: (project_id, url, callback = (error) ->) ->
fs.unlink UrlCache._cacheFilePathForUrl(project_id, url), callback
fs.unlink UrlCache._cacheFilePathForUrl(project_id, url), (error) ->
if error? and error.code != 'ENOENT' # no error if the file isn't present
return callback(error)
else
return callback()
_findUrlDetails: (project_id, url, callback = (error, urlDetails) ->) ->
db.UrlCache.find(where: { url: url, project_id: project_id })
.success((urlDetails) -> callback null, urlDetails)
.then((urlDetails) -> callback null, urlDetails)
.error callback
_updateOrCreateUrlDetails: (project_id, url, lastModified, callback = (error) ->) ->
db.UrlCache.findOrCreate(url: url, project_id: project_id)
.success(
(urlDetails) ->
db.UrlCache.findOrCreate(where: {url: url, project_id: project_id})
.spread(
(urlDetails, created) ->
urlDetails.updateAttributes(lastModified: lastModified)
.success(() -> callback())
.then(() -> callback())
.error(callback)
)
.error callback
_clearUrlDetails: (project_id, url, callback = (error) ->) ->
db.UrlCache.destroy(url: url, project_id: project_id)
.success(() -> callback null)
db.UrlCache.destroy(where: {url: url, project_id: project_id})
.then(() -> callback null)
.error callback
_findAllUrlsInProject: (project_id, callback = (error, urls) ->) ->
db.UrlCache.findAll(where: { project_id: project_id })
.success(
.then(
(urlEntries) ->
callback null, urlEntries.map((entry) -> entry.url)
)

View File

@@ -1,23 +1,65 @@
request = require("request").defaults(jar: false)
fs = require("fs")
logger = require "logger-sharelatex"
oneMinute = 60 * 1000
module.exports = UrlFetcher =
pipeUrlToFile: (url, filePath, _callback = (error) ->) ->
callbackOnce = (error) ->
clearTimeout timeoutHandler if timeoutHandler?
_callback(error)
_callback = () ->
urlStream = request.get(url)
fileStream = fs.createWriteStream(filePath)
timeoutHandler = setTimeout () ->
timeoutHandler = null
logger.error url:url, filePath: filePath, "Timed out downloading file to cache"
callbackOnce(new Error("Timed out downloading file to cache #{url}"))
# FIXME: maybe need to close fileStream here
, 3 * oneMinute
urlStream.on "response", (res) ->
if res.statusCode >= 200 and res.statusCode < 300
urlStream.pipe(fileStream)
else
callbackOnce(new Error("URL returned non-success status code: #{res.statusCode} #{url}"))
logger.log url:url, filePath: filePath, "started downloading url to cache"
urlStream = request.get({url: url, timeout: oneMinute})
urlStream.pause() # stop data flowing until we are ready
# attach handlers before setting up pipes
urlStream.on "error", (error) ->
logger.error err: error, url:url, filePath: filePath, "error downloading url"
callbackOnce(error or new Error("Something went wrong downloading the URL #{url}"))
urlStream.on "end", () ->
callbackOnce()
logger.log url:url, filePath: filePath, "finished downloading file into cache"
urlStream.on "response", (res) ->
if res.statusCode >= 200 and res.statusCode < 300
fileStream = fs.createWriteStream(filePath)
# attach handlers before setting up pipes
fileStream.on 'error', (error) ->
logger.error err: error, url:url, filePath: filePath, "error writing file into cache"
fs.unlink filePath, (err) ->
if err?
logger.err err: err, filePath: filePath, "error deleting file from cache"
callbackOnce(error)
fileStream.on 'finish', () ->
logger.log url:url, filePath: filePath, "finished writing file into cache"
callbackOnce()
fileStream.on 'pipe', () ->
logger.log url:url, filePath: filePath, "piping into filestream"
urlStream.pipe(fileStream)
urlStream.resume() # now we are ready to handle the data
else
logger.error statusCode: res.statusCode, url:url, filePath: filePath, "unexpected status code downloading url to cache"
# https://nodejs.org/api/http.html#http_class_http_clientrequest
# If you add a 'response' event handler, then you must consume
# the data from the response object, either by calling
# response.read() whenever there is a 'readable' event, or by
# adding a 'data' handler, or by calling the .resume()
# method. Until the data is consumed, the 'end' event will not
# fire. Also, until the data is read it will consume memory
# that can eventually lead to a 'process out of memory' error.
urlStream.resume() # discard the data
callbackOnce(new Error("URL returned non-success status code: #{res.statusCode} #{url}"))

View File

@@ -1,11 +1,14 @@
Sequelize = require("sequelize")
Settings = require("settings-sharelatex")
_ = require("underscore")
options = _.extend {logging:false}, Settings.mysql.clsi
sequelize = new Sequelize(
Settings.mysql.clsi.database,
Settings.mysql.clsi.username,
Settings.mysql.clsi.password,
Settings.mysql.clsi
options
)
module.exports =
@@ -13,11 +16,20 @@ module.exports =
url: Sequelize.STRING
project_id: Sequelize.STRING
lastModified: Sequelize.DATE
}, {
indexes: [
{fields: ['url', 'project_id']},
{fields: ['project_id']}
]
})
Project: sequelize.define("Project", {
project_id: Sequelize.STRING
project_id: {type: Sequelize.STRING, primaryKey: true}
lastAccessed: Sequelize.DATE
}, {
indexes: [
{fields: ['lastAccessed']}
]
})
sync: () -> sequelize.sync()

View File

@@ -16,24 +16,29 @@ module.exports =
clsiCacheDir: Path.resolve(__dirname + "/../cache")
synctexBaseDir: (project_id) -> Path.join(@compilesDir, project_id)
# clsi:
# commandRunner: "docker-runner-sharelatex"
# docker:
# image: "quay.io/sharelatex/texlive-full"
# env:
# PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/texlive/2013/bin/x86_64-linux/"
# HOME: "/tmp"
# modem:
# socketPath: false
# user: "tex"
internal:
clsi:
port: 3013
host: "localhost"
host: process.env["LISTEN_ADDRESS"] or "localhost"
apis:
clsi:
url: "http://localhost:3013"
smokeTest: false
project_cache_length_ms: 1000 * 60 * 60 * 24
parallelFileDownloads:1
if process.env["COMMAND_RUNNER"]
module.exports.clsi =
commandRunner: process.env["COMMAND_RUNNER"]
docker:
image: process.env["TEXLIVE_IMAGE"] or "quay.io/sharelatex/texlive-full:2017.1"
env:
HOME: "/tmp"
socketPath: "/var/run/docker.sock"
user: process.env["TEXLIVE_IMAGE_USER"] or "tex"
expireProjectAfterIdleMs: 24 * 60 * 60 * 1000
checkProjectsIntervalMs: 10 * 60 * 1000
module.exports.path.sandboxedCompilesHostDir = process.env["COMPILES_HOST_DIR"]

View File

@@ -1,27 +1,37 @@
{
"name": "node-clsi",
"description": "A Node.js implementation of the CLSI LaTeX web-API",
"version": "0.1.1",
"version": "0.1.4",
"repository": {
"type": "git",
"url": "https://github.com/sharelatex/clsi-sharelatex.git"
},
"scripts": {
"compile:app": "coffee -o app/js -c app/coffee && coffee -c app.coffee",
"start": "npm run compile:app && node app.js"
},
"author": "James Allen <james@sharelatex.com>",
"dependencies": {
"async": "0.2.9",
"lynx": "0.0.11",
"mkdirp": "0.3.5",
"mysql": "2.0.0-alpha7",
"request": "~2.21.0",
"logger-sharelatex": "git+https://github.com/sharelatex/logger-sharelatex.git#v1.0.0",
"settings-sharelatex": "git+https://github.com/sharelatex/settings-sharelatex.git#v1.0.0",
"metrics-sharelatex": "git+https://github.com/sharelatex/metrics-sharelatex.git#v1.0.0",
"sequelize": "2.0.0-beta.2",
"wrench": "~1.5.4",
"smoke-test-sharelatex": "git+https://github.com/sharelatex/smoke-test-sharelatex.git#v1.0.0",
"sqlite3": "~2.2.0",
"body-parser": "^1.2.0",
"express": "^4.2.0",
"body-parser": "^1.2.0"
"fs-extra": "^0.16.3",
"grunt-mkdir": "^1.0.0",
"heapdump": "^0.3.5",
"lockfile": "^1.0.3",
"logger-sharelatex": "git+https://github.com/sharelatex/logger-sharelatex.git#v1.5.4",
"lynx": "0.0.11",
"metrics-sharelatex": "git+https://github.com/sharelatex/metrics-sharelatex.git#v1.5.0",
"mkdirp": "0.3.5",
"mysql": "2.6.2",
"request": "^2.21.0",
"sequelize": "^2.1.3",
"settings-sharelatex": "git+https://github.com/sharelatex/settings-sharelatex.git#v1.0.0",
"smoke-test-sharelatex": "git+https://github.com/sharelatex/smoke-test-sharelatex.git#v0.2.0",
"sqlite3": "~3.1.8",
"underscore": "^1.8.2",
"v8-profiler": "^5.2.4",
"wrench": "~1.5.4"
},
"devDependencies": {
"mocha": "1.10.0",

View File

@@ -12,18 +12,35 @@ catch e
convertToPng = (pdfPath, pngPath, callback = (error) ->) ->
convert = ChildProcess.exec "convert #{fixturePath(pdfPath)} #{fixturePath(pngPath)}"
stdout = ""
convert.stdout.on "data", (chunk) -> console.log "STDOUT", chunk.toString()
convert.stderr.on "data", (chunk) -> console.log "STDERR", chunk.toString()
convert.on "exit", () ->
callback()
compare = (originalPath, generatedPath, callback = (error, same) ->) ->
proc = ChildProcess.exec "compare -metric mae #{fixturePath(originalPath)} #{fixturePath(generatedPath)} #{fixturePath("tmp/diff.png")}"
diff_file = "#{fixturePath(generatedPath)}-diff.png"
proc = ChildProcess.exec "compare -metric mae #{fixturePath(originalPath)} #{fixturePath(generatedPath)} #{diff_file}"
stderr = ""
proc.stderr.on "data", (chunk) -> stderr += chunk
proc.on "exit", () ->
if stderr.trim() == "0 (0)"
fs.unlink diff_file # remove output diff if test matches expected image
callback null, true
else
console.log stderr
console.log "compare result", stderr
callback null, false
checkPdfInfo = (pdfPath, callback = (error, output) ->) ->
proc = ChildProcess.exec "pdfinfo #{fixturePath(pdfPath)}"
stdout = ""
proc.stdout.on "data", (chunk) -> stdout += chunk
proc.stderr.on "data", (chunk) -> console.log "STDERR", chunk.toString()
proc.on "exit", () ->
if stdout.match(/Optimized:\s+yes/)
callback null, true
else
console.log "pdfinfo result", stdout
callback null, false
compareMultiplePages = (project_id, callback = (error) ->) ->
@@ -39,24 +56,30 @@ compareMultiplePages = (project_id, callback = (error) ->) ->
compareNext page_no + 1, callback
compareNext 0, callback
comparePdf = (project_id, example_dir, callback = (error) ->) ->
convertToPng "tmp/#{project_id}.pdf", "tmp/#{project_id}-generated.png", (error) =>
throw error if error?
convertToPng "examples/#{example_dir}/output.pdf", "tmp/#{project_id}-source.png", (error) =>
throw error if error?
fs.stat fixturePath("tmp/#{project_id}-source-0.png"), (error, stat) =>
if error?
compare "tmp/#{project_id}-source.png", "tmp/#{project_id}-generated.png", (error, same) =>
throw error if error?
same.should.equal true
callback()
else
compareMultiplePages project_id, (error) ->
throw error if error?
callback()
downloadAndComparePdf = (project_id, example_dir, url, callback = (error) ->) ->
writeStream = fs.createWriteStream(fixturePath("tmp/#{project_id}.pdf"))
request.get(url).pipe(writeStream)
writeStream.on "close", () =>
convertToPng "tmp/#{project_id}.pdf", "tmp/#{project_id}-generated.png", (error) =>
checkPdfInfo "tmp/#{project_id}.pdf", (error, optimised) =>
throw error if error?
convertToPng "examples/#{example_dir}/output.pdf", "tmp/#{project_id}-source.png", (error) =>
throw error if error?
fs.stat fixturePath("tmp/#{project_id}-source-0.png"), (error, stat) =>
if error?
compare "tmp/#{project_id}-source.png", "tmp/#{project_id}-generated.png", (error, same) =>
throw error if error?
same.should.equal true
callback()
else
compareMultiplePages project_id, (error) ->
throw error if error?
callback()
optimised.should.equal true
comparePdf project_id, example_dir, callback
Client.runServer(4242, fixturePath("examples"))
@@ -68,15 +91,19 @@ describe "Example Documents", ->
do (example_dir) ->
describe example_dir, ->
before ->
@project_id = Client.randomId()
@project_id = Client.randomId() + "_" + example_dir
it "should generate the correct pdf", (done) ->
Client.compileDirectory @project_id, fixturePath("examples"), example_dir, 4242, (error, res, body) =>
if error || body?.compile?.status is "failure"
console.log "DEBUG: error", error, "body", JSON.stringify(body)
pdf = Client.getOutputFile body, "pdf"
downloadAndComparePdf(@project_id, example_dir, pdf.url, done)
it "should generate the correct pdf on the second run as well", (done) ->
Client.compileDirectory @project_id, fixturePath("examples"), example_dir, 4242, (error, res, body) =>
if error || body?.compile?.status is "failure"
console.log "DEBUG: error", error, "body", JSON.stringify(body)
pdf = Client.getOutputFile body, "pdf"
downloadAndComparePdf(@project_id, example_dir, pdf.url, done)

View File

@@ -35,4 +35,3 @@ describe "Syncing", ->
code: [ { file: 'main.tex', line: 3, column: -1 } ]
)
done()

View File

@@ -6,13 +6,14 @@ describe "Timed out compile", ->
before (done) ->
@request =
options:
timeout: 0.01 #seconds
timeout: 1 #seconds
resources: [
path: "main.tex"
content: '''
\\documentclass{article}
\\begin{document}
Hello world
\\input{|"sleep 10"}
\\end{document}
'''
]

View File

@@ -0,0 +1,36 @@
Client = require "./helpers/Client"
request = require "request"
require("chai").should()
expect = require("chai").expect
path = require("path")
fs = require("fs")
describe "Syncing", ->
before (done) ->
@request =
resources: [
path: "main.tex"
content: fs.readFileSync(path.join(__dirname,"../fixtures/naugty_strings.txt"),"utf-8")
]
@project_id = Client.randomId()
Client.compile @project_id, @request, (@error, @res, @body) => done()
describe "wordcount file", ->
it "should return wordcount info", (done) ->
Client.wordcount @project_id, "main.tex", (error, result) ->
throw error if error?
expect(result).to.deep.equal(
texcount: {
encode: "utf8"
textWords: 2281
headWords: 2
outside: 0
headers: 2
elements: 0
mathInline: 6
mathDisplay: 0
errors: 0
messages: ""
}
)
done()

View File

@@ -30,7 +30,10 @@ module.exports = Client =
express = require("express")
app = express()
app.use express.static(directory)
app.listen(port, host)
app.listen(port, host).on "error", (error) ->
console.error "error starting server:", error.message
process.exit(1)
syncFromCode: (project_id, file, line, column, callback = (error, pdfPositions) ->) ->
request.get {
@@ -90,3 +93,12 @@ module.exports = Client =
@compile project_id, req, callback
wordcount: (project_id, file, callback = (error, pdfPositions) ->) ->
request.get {
url: "#{@host}/project/#{project_id}/wordcount"
qs: {
file: file
}
}, (error, response, body) ->
return callback(error) if error?
callback null, JSON.parse(body)

View File

@@ -0,0 +1,12 @@
\documentclass{article}
\usepackage{fontawesome}
\begin{document}
Cloud \faCloud
Cog \faCog
Database \faDatabase
Leaf \faLeaf
\end{document}

View File

@@ -0,0 +1,16 @@
\documentclass{article}
\usepackage{fontspec}
\defaultfontfeatures{Extension = .otf} % this is needed because
% fontawesome package loads by
% font name only
\usepackage{fontawesome}
\begin{document}
Cloud \faCloud
Cog \faCog
Database \faDatabase
Leaf \faLeaf
\end{document}

View File

@@ -0,0 +1,3 @@
{
"compiler": "xelatex"
}

View File

@@ -0,0 +1,14 @@
\documentclass{article}
\usepackage[utf8x]{inputenc}
\usepackage[hebrew,english]{babel}
\begin{document}
\selectlanguage{hebrew}
כדי לכתוב משהו באנגלית חייבים להשתמש במקרו הבא וכאן
ממשיכים לכתוב בעברית. טקסט נוסחאות תמיד יהיה בכיוון שמאל-לימין
\selectlanguage{english}
This is a test.
\end{document}

Binary file not shown.

View File

@@ -0,0 +1,35 @@
\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[spanish]{babel}
\begin{document}
\tableofcontents
\vspace{2cm} %Add a 2cm space
\begin{abstract}
Este es un breve resumen del contenido del
documento escrito en español.
\end{abstract}
\section{Sección Introductoria}
Esta es la primera sección, podemos agregar
algunos elementos adicionales y todo será
escrito correctamente. Más aún, si una palabra
es demaciado larga y tiene que ser truncada,
babel tratará de truncarla correctamente
dependiendo del idioma.
\section{Sección con teoremas}
Esta sección es para ver que pasa con los comandos
que definen texto
%% chunk options: cache this chunk
%% begin.rcode my-cache, cache=TRUE
% set.seed(123)
% x = runif(10)
% sd(x) # standard deviation
%% end.rcode
\end{document}

View File

@@ -1,4 +1,4 @@
\documentclass{article}
\documentclass[a4paper]{article}
\usepackage{graphicx}

View File

@@ -1,17 +0,0 @@
Section Title
-------------
* List item one
* List item two
: Sample grid table.
+---------------+---------------+--------------------+
| Fruit | Price | Advantages |
+===============+===============+====================+
| Bananas | $1.34 | - built-in wrapper |
| | | - bright color |
+---------------+---------------+--------------------+
| Oranges | $2.10 | - cures scurvy |
| | | - tasty |
+---------------+---------------+--------------------+

View File

@@ -1,9 +0,0 @@
\documentclass{article}
\usepackage{longtable}
\usepackage{booktabs, multicol, multirow}
\begin{document}
\input{chapters/chapter1}
\end{document}

View File

@@ -1,23 +0,0 @@
% Title
% Author
% Date
Chapter title
=============
Section Title
-------------
Hello world. Have a nice table:
: Sample grid table.
+---------------+---------------+--------------------+
| Fruit | Price | Advantages |
+===============+===============+====================+
| Bananas | $1.34 | - built-in wrapper |
| | | - bright color |
+---------------+---------------+--------------------+
| Oranges | $2.10 | - cures scurvy |
| | | - tasty |
+---------------+---------------+--------------------+

View File

@@ -0,0 +1,66 @@
\RequirePackage{luatex85}
\documentclass[tikz]{standalone}
\usepackage[compat=1.1.0]{tikz-feynman}
\begin{document}
\feynmandiagram [horizontal=a to b] {
i1 -- [fermion] a -- [fermion] i2,
a -- [photon] b,
f1 -- [fermion] b -- [fermion] f2,
};
\feynmandiagram [horizontal=a to b] {
i1 [particle=\(e^{-}\)] -- [fermion] a -- [fermion] i2 [particle=\(e^{+}\)],
a -- [photon, edge label=\(\gamma\), momentum'=\(k\)] b,
f1 [particle=\(\mu^{+}\)] -- [fermion] b -- [fermion] f2 [particle=\(\mu^{-}\)],
};
\feynmandiagram [large, vertical=e to f] {
a -- [fermion] b -- [photon, momentum=\(k\)] c -- [fermion] d,
b -- [fermion, momentum'=\(p_{1}\)] e -- [fermion, momentum'=\(p_{2}\)] c,
e -- [gluon] f,
h -- [fermion] f -- [fermion] i,
};
\begin{tikzpicture}
\begin{feynman}
\vertex (a1) {\(\overline b\)};
\vertex[right=1cm of a1] (a2);
\vertex[right=1cm of a2] (a3);
\vertex[right=1cm of a3] (a4) {\(b\)};
\vertex[right=1cm of a4] (a5);
\vertex[right=2cm of a5] (a6) {\(u\)};
\vertex[below=2em of a1] (b1) {\(d\)};
\vertex[right=1cm of b1] (b2);
\vertex[right=1cm of b2] (b3);
\vertex[right=1cm of b3] (b4) {\(\overline d\)};
\vertex[below=2em of a6] (b5) {\(\overline d\)};
\vertex[above=of a6] (c1) {\(\overline u\)};
\vertex[above=2em of c1] (c3) {\(d\)};
\vertex at ($(c1)!0.5!(c3) - (1cm, 0)$) (c2);
\diagram* {
{[edges=fermion]
(b1) -- (b2) -- (a2) -- (a1),
(b5) -- (b4) -- (b3) -- (a3) -- (a4) -- (a5) -- (a6),
},
(a2) -- [boson, edge label=\(W\)] (a3),
(b2) -- [boson, edge label'=\(W\)] (b3),
(c1) -- [fermion, out=180, in=-45] (c2) -- [fermion, out=45, in=180] (c3),
(a5) -- [boson, bend left, edge label=\(W^{-}\)] (c2),
};
\draw [decoration={brace}, decorate] (b1.south west) -- (a1.north west)
node [pos=0.5, left] {\(B^{0}\)};
\draw [decoration={brace}, decorate] (c3.north east) -- (c1.south east)
node [pos=0.5, right] {\(\pi^{-}\)};
\draw [decoration={brace}, decorate] (a6.north east) -- (b5.south east)
node [pos=0.5, right] {\(\pi^{+}\)};
\end{feynman}
\end{tikzpicture}
\end{document}

View File

@@ -0,0 +1,3 @@
{
"compiler": "lualatex"
}

View File

@@ -0,0 +1,626 @@
\documentclass{article}
\usepackage[utf8]{inputenc}
\title{eee}
\author{henry.oswald }
\date{September 2015}
\usepackage{natbib}
\usepackage{graphicx}
\begin{document}
\maketitle
\section{Introduction}
Encoding: utf8
# Reserved Strings
#
# Strings which may be used elsewhere in code
undefined
undef
null
NULL
(null)
nil
NIL
true
false
True
False
None
\
\\
# Numeric Strings
#
# Strings which can be interpreted as numeric
0
1
1.00
$1.00
1/2
1E2
1E02
1E+02
-1
-1.00
-$1.00
-1/2
-1E2
-1E02
-1E+02
1/0
0/0
-2147483648/-1
-9223372036854775808/-1
0.00
0..0
.
0.0.0
0,00
0,,0
,
0,0,0
0.0/0
1.0/0.0
0.0/0.0
1,0/0,0
0,0/0,0
--1
-
-.
-,
999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999
NaN
Infinity
-Infinity
0x0
0xffffffff
0xffffffffffffffff
0xabad1dea
123456789012345678901234567890123456789
1,000.00
1 000.00
1'000.00
1,000,000.00
1 000 000.00
1'000'000.00
1.000,00
1 000,00
1'000,00
1.000.000,00
1 000 000,00
1'000'000,00
01000
08
09
2.2250738585072011e-308
# Special Characters
#
# Strings which contain common special ASCII characters (may need to be escaped)
,./;'[]\-=
<>?:"{}|_+
!@#$%^&*()`~
# Unicode Symbols
#
# Strings which contain common unicode symbols (e.g. smart quotes)
Ω≈ç√∫˜µ≤≥÷
åß∂ƒ©˙∆˚¬…æ
œ∑´®†¥¨ˆøπ“‘
¡™£¢∞§¶•ªº–≠
¸˛Ç◊ı˜Â¯˘¿
ÅÍÎÏ˝ÓÔÒÚÆ☃
Œ„´‰ˇÁ¨ˆØ∏”’
`⁄€‹›fifl‡°·‚—±
⅛⅜⅝⅞
ЁЂЃЄЅІЇЈЉЊЋЌЍЎЏАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэюя
٠١٢٣٤٥٦٧٨٩
# Unicode Subscript/Superscript
#
# Strings which contain unicode subscripts/superscripts; can cause rendering issues
⁰⁴⁵
₀₁₂
⁰⁴⁵₀₁₂
# Quotation Marks
#
# Strings which contain misplaced quotation marks; can cause encoding errors
'
"
''
""
'"'
"''''"'"
"'"'"''''"
# Two-Byte Characters
#
# Strings which contain two-byte characters: can cause rendering issues or character-length issues
田中さんにあげて下さい
パーティーへ行かないか
和製漢語
部落格
사회과학원 어학연구소
찦차를 타고 온 펲시맨과 쑛다리 똠방각하
社會科學院語學研究所
울란바토르
𠜎𠜱𠝹𠱓𠱸𠲖𠳏
# Japanese Emoticons
#
# Strings which consists of Japanese-style emoticons which are popular on the web
ヽ༼ຈل͜ຈ༽ノ ヽ༼ຈل͜ຈ༽ノ
(。◕ ∀ ◕。)
`ィ(´∀`∩
__ロ(,_,*)
・( ̄∀ ̄)・:*:
゚・✿ヾ╲(。◕‿◕。)╱✿・゚
,。・:*:・゜’( ☻ ω ☻ )。・:*:・゜’
(╯°□°)╯︵ ┻━┻)
(ノಥ益ಥ)ノ ┻━┻
( ͡° ͜ʖ ͡°)
# Emoji
#
# Strings which contain Emoji; should be the same behavior as two-byte characters, but not always
😍
👩🏽
👾 🙇 💁 🙅 🙆 🙋 🙎 🙍
🐵 🙈 🙉 🙊
❤️ 💔 💌 💕 💞 💓 💗 💖 💘 💝 💟 💜 💛 💚 💙
✋🏿 💪🏿 👐🏿 🙌🏿 👏🏿 🙏🏿
🚾 🆒 🆓 🆕 🆖 🆗 🆙 🏧
0⃣ 1⃣ 2⃣ 3⃣ 4⃣ 5⃣ 6⃣ 7⃣ 8⃣ 9⃣ 🔟
# Unicode Numbers
#
# Strings which contain unicode numbers; if the code is localized, it should see the input as numeric
١٢٣
# Right-To-Left Strings
#
# Strings which contain text that should be rendered RTL if possible (e.g. Arabic, Hebrew)
ثم نفس سقطت وبالتحديد،, جزيرتي باستخدام أن دنو. إذ هنا؟ الستار وتنصيب كان. أهّل ايطاليا، بريطانيا-فرنسا قد أخذ. سليمان، إتفاقية بين ما, يذكر الحدود أي بعد, معاملة بولندا، الإطلاق عل إيو.
בְּרֵאשִׁית, בָּרָא אֱלֹהִים, אֵת הַשָּׁמַיִם, וְאֵת הָאָרֶץ
הָיְתָהtestالصفحات التّحول
# Unicode Spaces
#
# Strings which contain unicode space characters with special properties (c.f. https://www.cs.tut.fi/~jkorpela/chars/spaces.html)
 

# Trick Unicode
#
# Strings which contain unicode with unusual properties (e.g. Right-to-left override) (c.f. http://www.unicode.org/charts/PDF/U2000.pdf)
test
test
test
testtest
test
# Zalgo Text
#
# Strings which contain "corrupted" text. The corruption will not appear in non-HTML text, however. (via http://www.eeemo.net)
Ṱ̺̺̕o͞ ̷i̲̬͇̪͙n̝̗͕v̟̜̘̦͟o̶̙̰̠kè͚̮̺̪̹̱̤ ̖t̝͕̳̣̻̪͞h̼͓̲̦̳̘̲e͇̣̰̦̬͎ ̢̼̻̱̘h͚͎͙̜̣̲ͅi̦̲̣̰̤v̻͍e̺̭̳̪̰-m̢iͅn̖̺̞̲̯̰d̵̼̟͙̩̼̘̳ ̞̥̱̳̭r̛̗̘e͙p͠r̼̞̻̭̗e̺̠̣͟s̘͇̳͍̝͉e͉̥̯̞̲͚̬͜ǹ̬͎͎̟̖͇̤t͍̬̤͓̼̭͘ͅi̪̱n͠g̴͉ ͏͉ͅc̬̟h͡a̫̻̯͘o̫̟̖͍̙̝͉s̗̦̲.̨̹͈̣
̡͓̞ͅI̗̘̦͝n͇͇͙v̮̫ok̲̫̙͈i̖͙̭̹̠̞n̡̻̮̣̺g̲͈͙̭͙̬͎ ̰t͔̦h̞̲e̢̤ ͍̬̲͖f̴̘͕̣è͖ẹ̥̩l͖͔͚i͓͚̦͠n͖͍̗͓̳̮g͍ ̨o͚̪͡f̘̣̬ ̖̘͖̟͙̮c҉͔̫͖͓͇͖ͅh̵̤̣͚͔á̗̼͕ͅo̼̣̥s̱͈̺̖̦̻͢.̛̖̞̠̫̰
̗̺͖̹̯͓Ṯ̤͍̥͇͈h̲́e͏͓̼̗̙̼̣͔ ͇̜̱̠͓͍ͅN͕͠e̗̱z̘̝̜̺͙p̤̺̹͍̯͚e̠̻̠͜r̨̤͍̺̖͔̖̖d̠̟̭̬̝͟i̦͖̩͓͔̤a̠̗̬͉̙n͚͜ ̻̞̰͚ͅh̵͉i̳̞v̢͇ḙ͎͟-҉̭̩̼͔m̤̭̫i͕͇̝̦n̗͙ḍ̟ ̯̲͕͞ǫ̟̯̰̲͙̻̝f ̪̰̰̗̖̭̘͘c̦͍̲̞͍̩̙ḥ͚a̮͎̟̙͜ơ̩̹͎s̤.̝̝ ҉Z̡̖̜͖̰̣͉̜a͖̰͙̬͡l̲̫̳͍̩g̡̟̼̱͚̞̬ͅo̗͜.̟
̦H̬̤̗̤͝e͜ ̜̥̝̻͍̟́w̕h̖̯͓o̝͙̖͎̱̮ ҉̺̙̞̟͈W̷̼̭a̺̪͍į͈͕̭͙̯̜t̶̼̮s̘͙͖̕ ̠̫̠B̻͍͙͉̳ͅe̵h̵̬͇̫͙i̹͓̳̳̮͎̫̕n͟d̴̪̜̖ ̰͉̩͇͙̲͞ͅT͖̼͓̪͢h͏͓̮̻e̬̝̟ͅ ̤̹̝W͙̞̝͔͇͝ͅa͏͓͔̹̼̣l̴͔̰̤̟͔ḽ̫.͕
Z̮̞̠͙͔ͅḀ̗̞͈̻̗Ḷ͙͎̯̹̞͓G̻O̭̗̮
# Unicode Upsidedown
#
# Strings which contain unicode with an "upsidedown" effect (via http://www.upsidedowntext.com)
˙ɐnbᴉlɐ ɐuƃɐɯ ǝɹolop ʇǝ ǝɹoqɐl ʇn ʇunpᴉpᴉɔuᴉ ɹodɯǝʇ poɯsnᴉǝ op pǝs 'ʇᴉlǝ ƃuᴉɔsᴉdᴉpɐ ɹnʇǝʇɔǝsuoɔ 'ʇǝɯɐ ʇᴉs ɹolop ɯnsdᴉ ɯǝɹo˥
00˙Ɩ$-
# Unicode font
#
# Strings which contain bold/italic/etc. versions of normal characters
𝐓𝐡𝐞 𝐪𝐮𝐢𝐜𝐤 𝐛𝐫𝐨𝐰𝐧 𝐟𝐨𝐱 𝐣𝐮𝐦𝐩𝐬 𝐨𝐯𝐞𝐫 𝐭𝐡𝐞 𝐥𝐚𝐳𝐲 𝐝𝐨𝐠
𝕿𝖍𝖊 𝖖𝖚𝖎𝖈𝖐 𝖇𝖗𝖔𝖜𝖓 𝖋𝖔𝖝 𝖏𝖚𝖒𝖕𝖘 𝖔𝖛𝖊𝖗 𝖙𝖍𝖊 𝖑𝖆𝖟𝖞 𝖉𝖔𝖌
𝑻𝒉𝒆 𝒒𝒖𝒊𝒄𝒌 𝒃𝒓𝒐𝒘𝒏 𝒇𝒐𝒙 𝒋𝒖𝒎𝒑𝒔 𝒐𝒗𝒆𝒓 𝒕𝒉𝒆 𝒍𝒂𝒛𝒚 𝒅𝒐𝒈
𝓣𝓱𝓮 𝓺𝓾𝓲𝓬𝓴 𝓫𝓻𝓸𝔀𝓷 𝓯𝓸𝔁 𝓳𝓾𝓶𝓹𝓼 𝓸𝓿𝓮𝓻 𝓽𝓱𝓮 𝓵𝓪𝔃𝔂 𝓭𝓸𝓰
𝕋𝕙𝕖 𝕢𝕦𝕚𝕔𝕜 𝕓𝕣𝕠𝕨𝕟 𝕗𝕠𝕩 𝕛𝕦𝕞𝕡𝕤 𝕠𝕧𝕖𝕣 𝕥𝕙𝕖 𝕝𝕒𝕫𝕪 𝕕𝕠𝕘
𝚃𝚑𝚎 𝚚𝚞𝚒𝚌𝚔 𝚋𝚛𝚘𝚠𝚗 𝚏𝚘𝚡 𝚓𝚞𝚖𝚙𝚜 𝚘𝚟𝚎𝚛 𝚝𝚑𝚎 𝚕𝚊𝚣𝚢 𝚍𝚘𝚐
⒯⒣⒠ ⒬⒰⒤⒞⒦ ⒝⒭⒪⒲⒩ ⒡⒪⒳ ⒥⒰⒨⒫⒮ ⒪⒱⒠⒭ ⒯⒣⒠ ⒧⒜⒵⒴ ⒟⒪⒢
# Script Injection
#
# Strings which attempt to invoke a benign script injection; shows vulnerability to XSS
<script>alert(123)</script>
&lt;script&gt;alert(&#39;123&#39;);&lt;/script&gt;
<img src=x onerror=alert(123) />
<svg><script>123<1>alert(123)</script>
"><script>alert(123)</script>
'><script>alert(123)</script>
><script>alert(123)</script>
</script><script>alert(123)</script>
< / script >< script >alert(123)< / script >
onfocus=JaVaSCript:alert(123) autofocus
" onfocus=JaVaSCript:alert(123) autofocus
' onfocus=JaVaSCript:alert(123) autofocus
scriptalert(123)/script
<sc<script>ript>alert(123)</sc</script>ript>
--><script>alert(123)</script>
";alert(123);t="
';alert(123);t='
JavaSCript:alert(123)
;alert(123);
src=JaVaSCript:prompt(132)
"><script>alert(123);</script x="
'><script>alert(123);</script x='
><script>alert(123);</script x=
" autofocus onkeyup="javascript:alert(123)
' autofocus onkeyup='javascript:alert(123)
<script\x20type="text/javascript">javascript:alert(1);</script>
<script\x3Etype="text/javascript">javascript:alert(1);</script>
<script\x0Dtype="text/javascript">javascript:alert(1);</script>
<script\x09type="text/javascript">javascript:alert(1);</script>
<script\x0Ctype="text/javascript">javascript:alert(1);</script>
<script\x2Ftype="text/javascript">javascript:alert(1);</script>
<script\x0Atype="text/javascript">javascript:alert(1);</script>
'`"><\x3Cscript>javascript:alert(1)</script>
'`"><\x00script>javascript:alert(1)</script>
ABC<div style="x\x3Aexpression(javascript:alert(1)">DEF
ABC<div style="x:expression\x5C(javascript:alert(1)">DEF
ABC<div style="x:expression\x00(javascript:alert(1)">DEF
ABC<div style="x:exp\x00ression(javascript:alert(1)">DEF
ABC<div style="x:exp\x5Cression(javascript:alert(1)">DEF
ABC<div style="x:\x0Aexpression(javascript:alert(1)">DEF
ABC<div style="x:\x09expression(javascript:alert(1)">DEF
ABC<div style="x:\xE3\x80\x80expression(javascript:alert(1)">DEF
ABC<div style="x:\xE2\x80\x84expression(javascript:alert(1)">DEF
ABC<div style="x:\xC2\xA0expression(javascript:alert(1)">DEF
ABC<div style="x:\xE2\x80\x80expression(javascript:alert(1)">DEF
ABC<div style="x:\xE2\x80\x8Aexpression(javascript:alert(1)">DEF
ABC<div style="x:\x0Dexpression(javascript:alert(1)">DEF
ABC<div style="x:\x0Cexpression(javascript:alert(1)">DEF
ABC<div style="x:\xE2\x80\x87expression(javascript:alert(1)">DEF
ABC<div style="x:\xEF\xBB\xBFexpression(javascript:alert(1)">DEF
ABC<div style="x:\x20expression(javascript:alert(1)">DEF
ABC<div style="x:\xE2\x80\x88expression(javascript:alert(1)">DEF
ABC<div style="x:\x00expression(javascript:alert(1)">DEF
ABC<div style="x:\xE2\x80\x8Bexpression(javascript:alert(1)">DEF
ABC<div style="x:\xE2\x80\x86expression(javascript:alert(1)">DEF
ABC<div style="x:\xE2\x80\x85expression(javascript:alert(1)">DEF
ABC<div style="x:\xE2\x80\x82expression(javascript:alert(1)">DEF
ABC<div style="x:\x0Bexpression(javascript:alert(1)">DEF
ABC<div style="x:\xE2\x80\x81expression(javascript:alert(1)">DEF
ABC<div style="x:\xE2\x80\x83expression(javascript:alert(1)">DEF
ABC<div style="x:\xE2\x80\x89expression(javascript:alert(1)">DEF
<a href="\x0Bjavascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x0Fjavascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xC2\xA0javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x05javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE1\xA0\x8Ejavascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x18javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x11javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE2\x80\x88javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE2\x80\x89javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE2\x80\x80javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x17javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x03javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x0Ejavascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x1Ajavascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x00javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x10javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE2\x80\x82javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x20javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x13javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x09javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE2\x80\x8Ajavascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x14javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x19javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE2\x80\xAFjavascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x1Fjavascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE2\x80\x81javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x1Djavascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE2\x80\x87javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x07javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE1\x9A\x80javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE2\x80\x83javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x04javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x01javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x08javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE2\x80\x84javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE2\x80\x86javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE3\x80\x80javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x12javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x0Djavascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x0Ajavascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x0Cjavascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x15javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE2\x80\xA8javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x16javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x02javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x1Bjavascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x06javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE2\x80\xA9javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE2\x80\x85javascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x1Ejavascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\xE2\x81\x9Fjavascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="\x1Cjavascript:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="javascript\x00:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="javascript\x3A:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="javascript\x09:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="javascript\x0D:javascript:alert(1)" id="fuzzelement1">test</a>
<a href="javascript\x0A:javascript:alert(1)" id="fuzzelement1">test</a>
`"'><img src=xxx:x \x0Aonerror=javascript:alert(1)>
`"'><img src=xxx:x \x22onerror=javascript:alert(1)>
`"'><img src=xxx:x \x0Bonerror=javascript:alert(1)>
`"'><img src=xxx:x \x0Donerror=javascript:alert(1)>
`"'><img src=xxx:x \x2Fonerror=javascript:alert(1)>
`"'><img src=xxx:x \x09onerror=javascript:alert(1)>
`"'><img src=xxx:x \x0Conerror=javascript:alert(1)>
`"'><img src=xxx:x \x00onerror=javascript:alert(1)>
`"'><img src=xxx:x \x27onerror=javascript:alert(1)>
`"'><img src=xxx:x \x20onerror=javascript:alert(1)>
"`'><script>\x3Bjavascript:alert(1)</script>
"`'><script>\x0Djavascript:alert(1)</script>
"`'><script>\xEF\xBB\xBFjavascript:alert(1)</script>
"`'><script>\xE2\x80\x81javascript:alert(1)</script>
"`'><script>\xE2\x80\x84javascript:alert(1)</script>
"`'><script>\xE3\x80\x80javascript:alert(1)</script>
"`'><script>\x09javascript:alert(1)</script>
"`'><script>\xE2\x80\x89javascript:alert(1)</script>
"`'><script>\xE2\x80\x85javascript:alert(1)</script>
"`'><script>\xE2\x80\x88javascript:alert(1)</script>
"`'><script>\x00javascript:alert(1)</script>
"`'><script>\xE2\x80\xA8javascript:alert(1)</script>
"`'><script>\xE2\x80\x8Ajavascript:alert(1)</script>
"`'><script>\xE1\x9A\x80javascript:alert(1)</script>
"`'><script>\x0Cjavascript:alert(1)</script>
"`'><script>\x2Bjavascript:alert(1)</script>
"`'><script>\xF0\x90\x96\x9Ajavascript:alert(1)</script>
"`'><script>-javascript:alert(1)</script>
"`'><script>\x0Ajavascript:alert(1)</script>
"`'><script>\xE2\x80\xAFjavascript:alert(1)</script>
"`'><script>\x7Ejavascript:alert(1)</script>
"`'><script>\xE2\x80\x87javascript:alert(1)</script>
"`'><script>\xE2\x81\x9Fjavascript:alert(1)</script>
"`'><script>\xE2\x80\xA9javascript:alert(1)</script>
"`'><script>\xC2\x85javascript:alert(1)</script>
"`'><script>\xEF\xBF\xAEjavascript:alert(1)</script>
"`'><script>\xE2\x80\x83javascript:alert(1)</script>
"`'><script>\xE2\x80\x8Bjavascript:alert(1)</script>
"`'><script>\xEF\xBF\xBEjavascript:alert(1)</script>
"`'><script>\xE2\x80\x80javascript:alert(1)</script>
"`'><script>\x21javascript:alert(1)</script>
"`'><script>\xE2\x80\x82javascript:alert(1)</script>
"`'><script>\xE2\x80\x86javascript:alert(1)</script>
"`'><script>\xE1\xA0\x8Ejavascript:alert(1)</script>
"`'><script>\x0Bjavascript:alert(1)</script>
"`'><script>\x20javascript:alert(1)</script>
"`'><script>\xC2\xA0javascript:alert(1)</script>
<img \x00src=x onerror="alert(1)">
<img \x47src=x onerror="javascript:alert(1)">
<img \x11src=x onerror="javascript:alert(1)">
<img \x12src=x onerror="javascript:alert(1)">
<img\x47src=x onerror="javascript:alert(1)">
<img\x10src=x onerror="javascript:alert(1)">
<img\x13src=x onerror="javascript:alert(1)">
<img\x32src=x onerror="javascript:alert(1)">
<img\x47src=x onerror="javascript:alert(1)">
<img\x11src=x onerror="javascript:alert(1)">
<img \x47src=x onerror="javascript:alert(1)">
<img \x34src=x onerror="javascript:alert(1)">
<img \x39src=x onerror="javascript:alert(1)">
<img \x00src=x onerror="javascript:alert(1)">
<img src\x09=x onerror="javascript:alert(1)">
<img src\x10=x onerror="javascript:alert(1)">
<img src\x13=x onerror="javascript:alert(1)">
<img src\x32=x onerror="javascript:alert(1)">
<img src\x12=x onerror="javascript:alert(1)">
<img src\x11=x onerror="javascript:alert(1)">
<img src\x00=x onerror="javascript:alert(1)">
<img src\x47=x onerror="javascript:alert(1)">
<img src=x\x09onerror="javascript:alert(1)">
<img src=x\x10onerror="javascript:alert(1)">
<img src=x\x11onerror="javascript:alert(1)">
<img src=x\x12onerror="javascript:alert(1)">
<img src=x\x13onerror="javascript:alert(1)">
<img[a][b][c]src[d]=x[e]onerror=[f]"alert(1)">
<img src=x onerror=\x09"javascript:alert(1)">
<img src=x onerror=\x10"javascript:alert(1)">
<img src=x onerror=\x11"javascript:alert(1)">
<img src=x onerror=\x12"javascript:alert(1)">
<img src=x onerror=\x32"javascript:alert(1)">
<img src=x onerror=\x00"javascript:alert(1)">
<a href=java&#1&#2&#3&#4&#5&#6&#7&#8&#11&#12script:javascript:alert(1)>XXX</a>
<img src="x` `<script>javascript:alert(1)</script>"` `>
<img src onerror /" '"= alt=javascript:alert(1)//">
<title onpropertychange=javascript:alert(1)></title><title title=>
<a href=http://foo.bar/#x=`y></a><img alt="`><img src=x:x onerror=javascript:alert(1)></a>">
<!--[if]><script>javascript:alert(1)</script -->
<!--[if<img src=x onerror=javascript:alert(1)//]> -->
<script src="/\%(jscript)s"></script>
<script src="\\%(jscript)s"></script>
<IMG """><SCRIPT>alert("XSS")</SCRIPT>">
<IMG SRC=javascript:alert(String.fromCharCode(88,83,83))>
<IMG SRC=# onmouseover="alert('xxs')">
<IMG SRC= onmouseover="alert('xxs')">
<IMG onmouseover="alert('xxs')">
<IMG SRC=&#106;&#97;&#118;&#97;&#115;&#99;&#114;&#105;&#112;&#116;&#58;&#97;&#108;&#101;&#114;&#116;&#40;&#39;&#88;&#83;&#83;&#39;&#41;>
<IMG SRC=&#0000106&#0000097&#0000118&#0000097&#0000115&#0000099&#0000114&#0000105&#0000112&#0000116&#0000058&#0000097&#0000108&#0000101&#0000114&#0000116&#0000040&#0000039&#0000088&#0000083&#0000083&#0000039&#0000041>
<IMG SRC=&#x6A&#x61&#x76&#x61&#x73&#x63&#x72&#x69&#x70&#x74&#x3A&#x61&#x6C&#x65&#x72&#x74&#x28&#x27&#x58&#x53&#x53&#x27&#x29>
<IMG SRC="jav ascript:alert('XSS');">
<IMG SRC="jav&#x09;ascript:alert('XSS');">
<IMG SRC="jav&#x0A;ascript:alert('XSS');">
<IMG SRC="jav&#x0D;ascript:alert('XSS');">
perl -e 'print "<IMG SRC=java\0script:alert(\"XSS\")>";' > out
<IMG SRC=" &#14; javascript:alert('XSS');">
<SCRIPT/XSS SRC="http://ha.ckers.org/xss.js"></SCRIPT>
<BODY onload!#$%&()*~+-_.,:;?@[/|\]^`=alert("XSS")>
<SCRIPT/SRC="http://ha.ckers.org/xss.js"></SCRIPT>
<<SCRIPT>alert("XSS");//<</SCRIPT>
<SCRIPT SRC=http://ha.ckers.org/xss.js?< B >
<SCRIPT SRC=//ha.ckers.org/.j>
<IMG SRC="javascript:alert('XSS')"
<iframe src=http://ha.ckers.org/scriptlet.html <
\";alert('XSS');//
<plaintext>
# SQL Injection
#
# Strings which can cause a SQL injection if inputs are not sanitized
1;DROP TABLE users
1'; DROP TABLE users-- 1
' OR 1=1 -- 1
' OR '1'='1
# Server Code Injection
#
# Strings which can cause user to run code on server as a privileged user (c.f. https://news.ycombinator.com/item?id=7665153)
-
--
--version
--help
$USER
/dev/null; touch /tmp/blns.fail ; echo
`touch /tmp/blns.fail`
$(touch /tmp/blns.fail)
@{[system "touch /tmp/blns.fail"]}
# Command Injection (Ruby)
#
# Strings which can call system commands within Ruby/Rails applications
eval("puts 'hello world'")
System("ls -al /")
`ls -al /`
Kernel.exec("ls -al /")
Kernel.exit(1)
%x('ls -al /')
# XXE Injection (XML)
#
# String which can reveal system files when parsed by a badly configured XML parser
<?xml version="1.0" encoding="ISO-8859-1"?><!DOCTYPE foo [ <!ELEMENT foo ANY ><!ENTITY xxe SYSTEM "file:///etc/passwd" >]><foo>&xxe;</foo>
# Unwanted Interpolation
#
# Strings which can be accidentally expanded into different strings if evaluated in the wrong context, e.g. used as a printf format string or via Perl or shell eval. Might expose sensitive data from the program doing the interpolation, or might just represent the wrong string.
$HOME
$ENV{'HOME'}
%d
%s
%*.*s
# File Inclusion
#
# Strings which can cause user to pull in files that should not be a part of a web server
../../../../../../../../../../../etc/passwd%00
../../../../../../../../../../../etc/hosts
# Known CVEs and Vulnerabilities
#
# Strings that test for known vulnerabilities
() { 0; }; touch /tmp/blns.shellshock1.fail;
() { _; } >_[$($())] { touch /tmp/blns.shellshock2.fail; }
# MSDOS/Windows Special Filenames
#
# Strings which are reserved characters in MSDOS/Windows
CON
PRN
AUX
CLOCK$
NUL
A:
ZZ:
COM1
LPT1
LPT2
LPT3
COM2
COM3
COM4
# Scunthorpe Problem
#
# Innocuous strings which may be blocked by profanity filters (https://en.wikipedia.org/wiki/Scunthorpe_problem)
Scunthorpe General Hospital
Penistone Community Church
Lightwater Country Park
Jimmy Clitheroe
Horniman Museum
shitake mushrooms
RomansInSussex.co.uk
http://www.cum.qc.ca/
Craig Cockburn, Software Specialist
Linda Callahan
Dr. Herman I. Libshitz
magna cum laude
Super Bowl XXX
medieval erection of parapets
evaluate
mocha
expression
Arsenal canal
classic
Tyson Gay
# Human injection
#
# Strings which may cause human to reinterpret worldview
If you're reading this, you've been in a coma for almost 20 years now. We're trying a new technique. We don't know where this message will end up in your dream, but we hope it works. Please wake up, we miss you.
# Terminal escape codes
#
# Strings which punish the fools who use cat/type on this file
Roses are red, violets are blue. Hope you enjoy terminal hue
But now...for my greatest trick...
The quick brown fox... [Beeeep]
# iOS Vulnerability
#
# Strings which crashed iMessage in iOS versions 8.3 and earlier
Powerلُلُصّبُلُلصّبُررً ॣ ॣh ॣ ॣ冗
\end{document}

View File

@@ -0,0 +1,23 @@
#!/bin/bash -x
export SHARELATEX_CONFIG=`pwd`/test/acceptance/scripts/settings.test.coffee
echo ">> Starting server..."
grunt --no-color >server.log 2>&1 &
echo ">> Server started"
sleep 5
echo ">> Running acceptance tests..."
grunt --no-color mochaTest:acceptance
_test_exit_code=$?
echo ">> Killing server"
kill %1
echo ">> Done"
exit $_test_exit_code

View File

@@ -0,0 +1,47 @@
Path = require "path"
module.exports =
# Options are passed to Sequelize.
# See http://sequelizejs.com/documentation#usage-options for details
mysql:
clsi:
database: "clsi"
username: "clsi"
password: null
dialect: "sqlite"
storage: Path.resolve("db.sqlite")
path:
compilesDir: Path.resolve(__dirname + "/../../../compiles")
clsiCacheDir: Path.resolve(__dirname + "/../../../cache")
#synctexBaseDir: (project_id) -> Path.join(@compilesDir, project_id)
synctexBaseDir: () -> "/compile"
sandboxedCompilesHostDir: process.env['SANDBOXED_COMPILES_HOST_DIR']
clsi:
#strace: true
#archive_logs: true
commandRunner: "docker-runner-sharelatex"
latexmkCommandPrefix: ["/usr/bin/time", "-v"] # on Linux
docker:
image: process.env.TEXLIVE_IMAGE || "texlive-full:2017.1-opt"
env:
PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/texlive/2017/bin/x86_64-linux/"
HOME: "/tmp"
modem:
socketPath: false
user: process.env.SIBLING_CONTAINER_USER ||"111"
internal:
clsi:
port: 3013
load_port: 3044
host: "localhost"
apis:
clsi:
url: "http://localhost:3013"
smokeTest: false
project_cache_length_ms: 1000 * 60 * 60 * 24
parallelFileDownloads:1

234
test/load/coffee/bulk.tex Normal file
View File

@@ -0,0 +1,234 @@
Lorem ipsum dolor sit amet, consectetur adipiscing elit. In tincidunt mattis sapien at tempor. Mauris ac tristique erat. Praesent interdum ipsum sem, ac fermentum urna imperdiet in. Nulla tincidunt purus vitae ipsum sagittis tincidunt. Aenean id nisi ullamcorper, ultrices mi vel, iaculis nunc. Sed vel varius metus, ac eleifend mauris. Donec sed orci fringilla, fermentum nulla vehicula, sodales purus.
Maecenas nulla quam, congue vitae pellentesque sed, bibendum eu felis. Vestibulum congue gravida diam, in venenatis nisl lacinia id. Nullam eget purus ac enim dignissim consectetur vel at dolor. Integer rhoncus nisl eu odio luctus, at placerat dolor congue. Fusce sodales molestie sem eget scelerisque. Sed eros tellus, tempor eu commodo nec, maximus imperdiet eros. Aliquam vulputate ligula non bibendum tempus. In commodo eros ante, ultrices condimentum purus finibus ut. Suspendisse at eleifend mauris, vitae tincidunt sapien. Curabitur orci ipsum, aliquet a cursus efficitur, lacinia ac ex. Integer lacinia bibendum dui ut ullamcorper. Curabitur in ultricies tellus, quis ullamcorper sem. Praesent sodales dui odio. Ut lacinia aliquet eros, ut maximus nisi. Donec sit amet dui a neque interdum dapibus.
Ut vulputate sem in lectus porttitor ullamcorper. Nulla ut urna vitae tellus posuere aliquam vitae in odio. Praesent placerat laoreet viverra. Curabitur lacinia est lectus, eget euismod nisi viverra eget. Aliquam facilisis lectus ut tincidunt mollis. Donec ut rhoncus lorem. Vivamus ultricies venenatis congue. Etiam non risus quis leo sodales lacinia. Phasellus commodo feugiat sem quis dignissim. Nunc augue dui, bibendum sed leo vitae, malesuada vulputate sem.
Quisque nec semper nulla. Etiam dictum blandit interdum. Morbi leo leo, scelerisque vel enim vel, egestas volutpat ligula. Maecenas ac elementum lacus. Duis molestie nunc id metus iaculis, in hendrerit massa egestas. Praesent feugiat tempor dui, sit amet ultrices dui elementum id. Suspendisse cursus accumsan diam, non imperdiet diam dapibus facilisis. Praesent blandit urna felis, eget sodales nisi dictum non. Cras finibus quis augue a venenatis. In pretium condimentum arcu, at vehicula ex gravida ut. Etiam congue urna ipsum, mattis interdum neque cursus bibendum.
Morbi felis orci, ultricies eget magna gravida, blandit condimentum erat. Curabitur convallis quam eros, eu porta diam ornare vitae. Interdum et malesuada fames ac ante ipsum primis in faucibus. Sed eleifend convallis massa, eget tristique dolor iaculis sed. Mauris id nunc erat. Donec semper rhoncus libero sit amet rhoncus. Suspendisse cursus suscipit augue quis fermentum. Sed in maximus erat.
Ut ultrices massa vitae lectus dictum fermentum. Cras vitae risus metus. Curabitur eleifend hendrerit dolor sit amet rutrum. Pellentesque pellentesque dolor ut felis vehicula pharetra. Nam id ante eget turpis vehicula interdum in vitae odio. Nullam nec orci interdum, commodo massa et, rutrum purus. Aenean vitae porta sem. Nam in lacinia turpis. Duis dui ligula, molestie quis sagittis sit amet, faucibus ac leo. Curabitur sit amet porta ligula. Integer et sollicitudin velit. Donec magna justo, ultricies eu nunc ut, rutrum aliquam orci. Sed in dignissim sem. Proin rutrum velit urna, eu tincidunt ipsum fermentum non. Morbi id cursus nisl.
Curabitur sed gravida ex, posuere laoreet orci. Morbi ac lacus quis tortor faucibus feugiat. Etiam fringilla lacinia libero. Duis varius sem vel lorem euismod luctus. Fusce tincidunt quis sem in ullamcorper. Ut luctus massa aliquam hendrerit finibus. Ut venenatis, neque eu hendrerit finibus, nisl tortor venenatis eros, in imperdiet leo est quis erat. Fusce luctus posuere massa, ut fermentum sapien blandit ut. Maecenas feugiat consequat lorem, eget sagittis elit vestibulum sit amet. Vivamus molestie ante ut turpis laoreet facilisis vitae eu diam. Integer a tempor tortor. In hac habitasse platea dictumst. Quisque arcu est, blandit eu justo sed, posuere congue nisi. Aliquam magna augue, convallis ac scelerisque vel, cursus eget dui. Nam rutrum auctor odio, vel sagittis ipsum gravida vel.
Etiam elementum placerat egestas. Morbi nec mi posuere, congue ligula eu, sagittis turpis. Fusce urna nisi, dapibus in pretium et, lobortis eu arcu. Curabitur ornare urna mauris, vitae varius nulla posuere in. Integer faucibus euismod dui, a venenatis massa vehicula sit amet. Donec fringilla tellus vitae ligula pretium mattis. Aliquam aliquet quam augue, a luctus orci euismod sed. Morbi tincidunt tincidunt nulla, eget elementum turpis congue id. Suspendisse pellentesque nulla leo, fermentum ultrices massa sollicitudin vel. Morbi vel nisl consectetur, pulvinar sapien a, accumsan diam. Morbi posuere auctor nibh, nec maximus ante tincidunt ac. Etiam ut erat consectetur, molestie est sit amet, pharetra nulla. Quisque varius vestibulum ex, eget feugiat enim molestie ac. Nulla quis imperdiet risus.
Nullam nec tempor arcu. Duis fringilla mi at magna dignissim, quis feugiat turpis lacinia. Nunc sed aliquet ipsum. Curabitur at dolor in dui posuere ornare a ut ex. Ut congue neque quis justo iaculis, ut accumsan odio condimentum. Donec sed tempus diam. Phasellus tincidunt malesuada dui, nec gravida justo volutpat vel. Praesent mi purus, sagittis in imperdiet sed, sodales eu turpis. Nullam rutrum non lacus ac imperdiet. Ut ultrices lorem at facilisis feugiat. Morbi eros enim, tristique at nisl ut, venenatis porttitor ligula. Nullam sed diam at nibh tristique consectetur. Phasellus iaculis justo nisi, ut interdum ante rutrum sit amet. Pellentesque finibus felis blandit metus pulvinar lacinia.
Aliquam erat volutpat. Nulla eu tortor sit amet tellus bibendum tristique eget consequat metus. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut in aliquet augue. Phasellus laoreet nulla convallis finibus vehicula. Fusce et urna dui. Duis vel porta nunc. Nunc condimentum, justo at efficitur dignissim, lorem diam elementum ex, at dictum lectus sapien ac neque. Aliquam lacinia et ipsum lacinia efficitur. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Phasellus convallis urna orci, et dictum sapien porta sit amet. Maecenas euismod dolor mattis sapien vestibulum pulvinar.
Vestibulum eget posuere purus, et viverra est. Nullam egestas massa et finibus semper. Vestibulum egestas porta ante eget maximus. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Pellentesque bibendum magna et fermentum consequat. Duis non arcu quis justo dignissim tempus at id diam. Praesent condimentum vel leo ac efficitur. Phasellus sollicitudin ipsum ut consectetur euismod. Proin diam eros, placerat sed dui ac, porttitor pellentesque nibh. Curabitur fermentum volutpat enim, in ullamcorper ipsum euismod et. Nunc a justo tortor. Phasellus libero nunc, consectetur ut dolor non, volutpat condimentum metus.
Ut tincidunt est sem, eu venenatis lectus pretium pretium. Vivamus venenatis, erat nec sollicitudin semper, justo nulla euismod dui, quis tempor libero lectus sit amet neque. Sed in iaculis ipsum. Quisque ultricies sed mi a consequat. Sed tincidunt ante ut turpis vehicula, sed fringilla ligula efficitur. Cras eget suscipit sapien. Ut sed malesuada est, ut tempor leo. Mauris dignissim turpis quis turpis placerat cursus. Vivamus dictum dui sed blandit aliquet.
Ut cursus, nulla eget ultricies tempor, magna enim aliquam libero, eget tempus mauris mauris ut elit. Nulla a mi quam. Integer ullamcorper ex et enim ornare efficitur. Vivamus tellus orci, pharetra in suscipit ac, ultrices sit amet sapien. Pellentesque pretium mauris vel orci accumsan, a hendrerit lectus sagittis. Mauris id nisi commodo, eleifend arcu in, vestibulum metus. Fusce vulputate gravida tincidunt. Nulla cursus non tortor ut tincidunt. Phasellus vel nisi tempus, fringilla lectus sed, ultricies erat. Ut gravida, enim id facilisis consequat, est nisi scelerisque magna, eget pharetra elit mi elementum ligula. Morbi hendrerit tortor eget velit rhoncus, consequat porta nisl aliquet. Nam diam turpis, ullamcorper vitae nisi eu, ultrices hendrerit magna. Vivamus eget pretium elit. Vivamus vitae odio sit amet libero hendrerit imperdiet.
Aenean pharetra ex eget lectus sodales placerat. Fusce quis orci vel est suscipit venenatis. Curabitur maximus, sem in tincidunt imperdiet, nisl lorem venenatis mauris, eget facilisis lectus mauris a eros. Nam luctus sem ac diam ultrices, eget vulputate tortor efficitur. Nunc fermentum condimentum lacus id faucibus. Nunc ut tellus pretium, mattis eros vitae, scelerisque felis. Aenean ligula nulla, vulputate id eros id, vestibulum vulputate odio. Nunc in elit id augue porttitor auctor sed vitae lacus. Integer enim orci, auctor at magna eget, viverra tempus risus. Nulla suscipit metus tortor, ultricies vestibulum odio euismod at. Etiam consequat diam ac leo dignissim vulputate. Donec lectus lorem, finibus sed purus ac, eleifend condimentum ipsum.
Fusce ornare metus vel dui scelerisque vehicula. Proin dictum sapien nec auctor congue. Nunc id erat sed velit facilisis tincidunt. In convallis eu diam id aliquam. Suspendisse eu nisl ante. Sed sit amet arcu non erat sagittis vehicula. Quisque pellentesque at lectus quis maximus. Nam mollis nulla interdum lobortis egestas. Fusce eu tellus eget libero pretium venenatis quis tristique justo. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Proin tempor suscipit enim, eget lacinia augue malesuada sit amet. Ut ornare massa in magna pulvinar sagittis. Etiam non risus mi. Aenean aliquam dui et risus egestas aliquet.
Aenean semper dui risus. Aenean consequat id elit a finibus. Sed vitae est sed arcu interdum maximus interdum in leo. Donec justo lorem, dictum sed placerat sit amet, eleifend in justo. Integer efficitur metus id interdum fringilla. Morbi et dui vitae libero consectetur fermentum quis sed quam. Sed interdum aliquam lorem, at blandit lectus fermentum a. Aliquam ac mollis felis, ut vulputate massa. Praesent convallis cursus eleifend. Donec non sem auctor, efficitur nisi ac, egestas libero. Nullam turpis lacus, dignissim eget pellentesque sed, fermentum ut ipsum. Vestibulum a posuere lacus, vitae rutrum neque. In hac habitasse platea dictumst. Sed vel maximus sem. Etiam dapibus risus et consectetur auctor. Phasellus vestibulum posuere sagittis.
Aliquam nec libero at velit rhoncus pretium. Curabitur tristique blandit orci id vestibulum. Praesent in tempus arcu. Vivamus in felis tellus. Nunc ac fermentum massa. Cras nisi mi, sollicitudin eu maximus vitae, sodales gravida lorem. Vivamus mollis metus id lectus rhoncus consequat. In dui tellus, vulputate sit amet purus vel, volutpat ornare turpis. Fusce vitae massa non ligula lobortis rhoncus eget id sapien. Sed nec tempus lectus. Proin tempor risus ipsum, fermentum suscipit felis cursus sit amet.
Maecenas ut dignissim ante, vitae ornare lorem. Fusce nec convallis eros, sed finibus urna. Proin ut finibus dolor. In non nunc sed dui aliquam suscipit. Etiam semper varius ex, sed venenatis sem gravida in. Interdum et malesuada fames ac ante ipsum primis in faucibus. Phasellus tempus aliquet placerat. Nam odio mauris, pharetra ac felis vel, ornare cursus nisl. Phasellus elit risus, finibus id ornare ut, scelerisque sed nisi. Curabitur aliquet, magna in finibus congue, dui libero auctor dui, ut fermentum metus enim vitae ex. Duis at elementum tellus.
Suspendisse laoreet luctus sem sit amet tempor. Vestibulum non lorem fringilla, maximus nisl vel, pulvinar enim. Suspendisse egestas elit et sem sagittis rhoncus. Morbi nulla augue, semper euismod ultricies quis, maximus et lorem. Nulla nec posuere justo. Ut blandit nisl vitae turpis varius finibus. Donec porttitor eros neque, id mollis neque tempus et. Maecenas a massa placerat, laoreet nisl vel, venenatis diam.
Phasellus at leo vel nisi aliquet placerat. Vestibulum luctus erat quis velit laoreet auctor. Aenean ultricies nulla tristique metus commodo, id fermentum justo tristique. Nullam ut tincidunt libero. Suspendisse volutpat, lacus ac congue ultricies, metus mi imperdiet magna, in maximus turpis ex eget leo. Sed lorem nibh, vestibulum id sodales ac, sagittis at elit. Curabitur purus nunc, sodales eget vehicula vitae, bibendum gravida diam. Nullam dignissim consequat pharetra. Nullam a diam consectetur, mollis odio sed, blandit lectus. Vestibulum eu velit id massa varius sagittis. Quisque tempor ante ac mauris rhoncus molestie. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas.
Ut sit amet euismod mi. Nulla facilisis est posuere, feugiat est et, dictum nulla. Proin eleifend ultricies nunc. Sed commodo justo nisi, id suscipit massa malesuada ut. Donec aliquam nibh tellus, vitae gravida lectus ultricies quis. Nam pulvinar lobortis erat sit amet convallis. Sed quis magna facilisis, tincidunt dui non, hendrerit nunc. Morbi egestas, risus fringilla fermentum porttitor, nunc velit viverra mi, non sodales augue arcu ac sapien. Duis blandit urna at nisl pellentesque semper. Nulla et malesuada nulla. Aenean tristique tortor odio, sit amet luctus odio aliquam id. Phasellus facilisis lorem vitae velit aliquam imperdiet. Cras faucibus dolor eget neque fringilla, ut mattis ex hendrerit. Integer molestie porttitor sagittis.
Pellentesque diam quam, auctor eget tristique eget, molestie sit amet est. Pellentesque a eros non dui gravida volutpat. Donec molestie blandit nunc ac interdum. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse lobortis, neque non aliquet convallis, lectus ex venenatis ex, quis malesuada massa erat non dolor. In tristique, enim eu ultrices ultricies, lectus ligula pretium orci, commodo cursus ante est vel odio. Sed quis accumsan purus. Nam fringilla ex ut urna vestibulum, et feugiat diam ultrices. Vivamus tempus felis ac quam blandit convallis.
Vestibulum eros erat, volutpat in est at, blandit pharetra sapien. Interdum et malesuada fames ac ante ipsum primis in faucibus. Aenean quis enim orci. Aliquam imperdiet vel arcu ac sagittis. Mauris vitae augue sed metus commodo ornare. Nulla malesuada tellus nisl, eu vestibulum ante mollis a. Sed sagittis euismod nunc, sit amet hendrerit tortor condimentum eu.
Praesent lacinia massa eget mi auctor placerat. Fusce porttitor augue lectus, a cursus ante dictum vel. Vestibulum ultrices vel mauris in fermentum. Nunc tincidunt non magna sed pharetra. Donec porttitor rutrum arcu, vitae tincidunt lacus suscipit ac. Aliquam lorem mauris, pulvinar non dignissim sed, pulvinar vitae dui. Donec id neque eu velit imperdiet lacinia nec eu magna. Ut a purus sit amet nulla venenatis vulputate. Integer vulputate est sem, iaculis porttitor mi mattis et. Phasellus condimentum ipsum eget tellus viverra, a tincidunt nunc feugiat. Praesent posuere aliquam ex et faucibus. Nullam pretium felis id mauris luctus, a luctus eros sodales.
Mauris et condimentum velit. Praesent id dignissim odio. Phasellus nisl velit, molestie sed nisi et, sollicitudin tempor nisi. Pellentesque lacus eros, ultricies non leo sit amet, porttitor ullamcorper ipsum. Vestibulum maximus lorem ac justo tempus imperdiet. Suspendisse rhoncus, mi in commodo tempus, orci turpis feugiat dui, nec facilisis arcu diam ut mauris. Vestibulum risus ligula, ornare non cursus vel, pellentesque non augue. Morbi eu gravida arcu. Nunc sed fermentum lacus. Nulla id quam aliquet, aliquet lacus in, rutrum metus. Duis tristique sodales risus vel interdum. Integer rhoncus nibh eget semper malesuada. Nunc sit amet ante diam. Fusce tincidunt aliquam ex, at lobortis tellus porttitor non. Vestibulum tincidunt iaculis dui vel scelerisque.
Aliquam sagittis mauris eget massa accumsan viverra. Pellentesque luctus sit amet augue ac scelerisque. Praesent imperdiet nisi dolor, sed malesuada est commodo at. Aenean vel leo eget felis tincidunt interdum. Fusce orci mauris, egestas eget lectus et, finibus consectetur urna. Donec ut dapibus elit, eu lacinia neque. Ut et accumsan nulla. Sed ullamcorper ligula purus, eu dapibus nunc auctor vel. Ut convallis consectetur dapibus. Curabitur eget porttitor felis. Maecenas pretium ac leo vitae volutpat. Donec in augue sit amet lorem efficitur dignissim.
Praesent iaculis tristique rutrum. Pellentesque id odio vel purus bibendum sodales suscipit id odio. Nullam ac velit imperdiet, imperdiet nisi sed, malesuada ipsum. Quisque varius dictum efficitur. Phasellus efficitur varius imperdiet. Aenean facilisis libero non augue porttitor, nec interdum felis imperdiet. Etiam et libero id elit commodo tincidunt. Nullam rutrum odio id rutrum tristique. Cras vehicula aliquet risus ac elementum. Duis nisl urna, commodo eget ante et, vehicula tempus lacus.
Mauris eu sapien sed erat auctor volutpat vel vel tortor. Aenean in commodo felis. Donec a dui a urna varius aliquet quis at nisi. Pellentesque et urna lacinia, commodo arcu at, laoreet lectus. Aliquam sodales, massa in convallis aliquam, dui orci eleifend arcu, a gravida mauris magna sed arcu. Ut ac lectus in risus feugiat lobortis. Nulla quis est eget dui pharetra ultricies eget at risus. Phasellus sagittis molestie ligula, eget egestas orci volutpat vitae.
Fusce nec finibus ligula, sed volutpat tortor. Sed placerat quam fringilla augue pharetra dictum. Proin ornare mi erat, eget sollicitudin ligula venenatis vitae. Aliquam semper sagittis urna rutrum pharetra. Vivamus lacinia mattis erat, vitae ultrices arcu. Maecenas id lacus eget justo imperdiet vehicula commodo a leo. Quisque vitae eros interdum, posuere ex ornare, tincidunt lectus.
Vestibulum hendrerit sed libero et bibendum. Sed ornare eu massa ut vestibulum. Curabitur imperdiet odio felis, at ullamcorper eros rhoncus nec. Cras commodo nisl eu augue iaculis posuere. Aliquam massa tortor, consectetur quis dui in, mollis dictum tellus. Fusce porttitor dapibus arcu. Fusce finibus pretium porttitor.
Proin dapibus viverra nisi. Cras ullamcorper purus et consequat fermentum. Duis imperdiet in dui in imperdiet. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam quis enim at ipsum ultricies auctor scelerisque nec nulla. Vivamus ut efficitur enim. Quisque dictum quam ac dui iaculis efficitur. Morbi at nulla convallis, condimentum tellus sit amet, dapibus nunc. Morbi metus felis, commodo sit amet justo id, finibus sagittis lorem. Nam nisi diam, fermentum dapibus varius in, convallis eu leo. Phasellus ut nunc orci. Sed tincidunt mauris in ante consequat, id bibendum libero volutpat. Aliquam a dictum libero. Etiam massa odio, congue ut lorem tincidunt, elementum egestas ligula.
Ut semper arcu a lectus interdum euismod. Curabitur nec ultrices neque. In eget sapien nulla. Pellentesque pellentesque faucibus urna id placerat. Aenean condimentum posuere interdum. Etiam vel tristique lorem, in dapibus urna. Vestibulum facilisis lobortis metus ac egestas. Vestibulum ultrices aliquet dui id efficitur. Sed a velit sed erat ultrices sodales suscipit a tortor. Nam mattis rhoncus augue et viverra. Praesent volutpat gravida enim quis sodales.
Nam placerat nisl a ullamcorper pharetra. Sed eu eros egestas, suscipit ante id, efficitur mi. Curabitur accumsan gravida pellentesque. Vestibulum urna risus, condimentum vel libero in, porta pharetra nisi. Duis eu feugiat neque, quis condimentum dolor. Suspendisse et elementum urna. Vivamus malesuada nisi eget blandit faucibus. Duis eu lorem ac est ultrices placerat nec nec elit. Nunc sed sagittis ligula. Vivamus gravida suscipit tellus nec euismod.
Ut posuere porta diam, vitae euismod erat egestas vitae. Aenean imperdiet quis quam eget dictum. Cras vulputate elit eu nibh scelerisque, vitae consectetur nisi malesuada. Praesent iaculis, neque nec tempor elementum, est mi egestas urna, nec commodo neque lacus vel mi. In a orci eu metus elementum tincidunt nec id tortor. Aenean augue augue, vulputate a porta quis, bibendum finibus augue. Nam condimentum ante ac congue ultrices. Praesent eu nisi eu enim accumsan scelerisque et id augue. Cras gravida dictum suscipit. Nulla tristique tempor lacus non eleifend. Curabitur sodales est in arcu accumsan, vel dignissim nunc blandit. Aenean sodales sodales lectus volutpat commodo. Maecenas venenatis accumsan nibh, sit amet semper risus ultrices non.
In blandit iaculis dolor sit amet convallis. Aliquam quis nisl sit amet augue semper vehicula. Sed aliquam vel ex vel condimentum. Nunc diam massa, mattis ac felis vel, cursus tincidunt ligula. Aliquam erat volutpat. Quisque faucibus in metus in tempus. Ut pharetra congue tellus. Vivamus est libero, fringilla vel elit ac, rhoncus fermentum arcu. Praesent tortor diam, mattis in varius commodo, lacinia accumsan neque. Integer nec luctus nibh. Duis tincidunt velit nisi, id porttitor turpis posuere in. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam finibus tortor lectus. Curabitur condimentum orci eget urna sollicitudin vehicula.
Donec sagittis mi lacus, quis rutrum sapien scelerisque sed. In quis interdum velit. Nulla eget tincidunt enim. Fusce viverra, sem pharetra ultricies laoreet, magna erat ornare lectus, a viverra mauris magna id mi. Quisque vitae pretium velit. Integer venenatis vel sapien non varius. Praesent eros neque, posuere sit amet posuere ut, posuere a sem. Vestibulum porttitor interdum posuere. Nam viverra felis dolor, eget ultrices lacus tincidunt a. Suspendisse elementum rhoncus tristique. Nam vehicula, odio eu porta ullamcorper, neque nunc pretium neque, ac vehicula mauris eros ac turpis. Aliquam augue nisl, pharetra non mauris id, finibus egestas massa.
Aliquam rhoncus tortor a nunc vulputate gravida. Phasellus aliquam lorem ipsum, a suscipit orci euismod ac. Curabitur fringilla orci in ante aliquam venenatis. Ut nec sollicitudin orci. Morbi consectetur massa nec lacus vestibulum commodo. Donec quis erat at nibh scelerisque interdum. Donec sed velit molestie purus volutpat tempus. Aenean consequat, massa vitae mollis eleifend, felis ante convallis ex, quis egestas libero nisi interdum dui. Maecenas aliquet nisi quis est dapibus posuere.
Phasellus lectus ex, finibus non orci et, suscipit fermentum orci. Vestibulum sed ligula non arcu facilisis feugiat. Praesent pellentesque eros quis eleifend tempus. In hac habitasse platea dictumst. Nulla accumsan suscipit risus, nec dignissim purus sollicitudin quis. Vestibulum vestibulum ligula non massa congue commodo. Aliquam velit ante, facilisis et aliquet non, imperdiet nec velit. Nunc vel elit felis.
Sed sed ex ut dui cursus consectetur. Phasellus laoreet velit lacinia dui placerat tincidunt. Nullam ornare sagittis quam ac pretium. Donec imperdiet velit quis ipsum placerat, vitae lacinia felis sagittis. Aenean vitae dui fermentum, laoreet lacus egestas, faucibus libero. Maecenas blandit blandit mi, et mattis lectus placerat sollicitudin. Aliquam at semper nulla.
Sed scelerisque lacus felis, et commodo libero tincidunt ac. Ut vel elit vel ex luctus lacinia ut et nisi. Sed ac tristique nisl. Suspendisse efficitur varius purus, sit amet gravida orci sagittis lacinia. Proin non placerat urna. Duis vehicula faucibus est vitae vehicula. Praesent vehicula tempor eros, in aliquet nisl vehicula in. Phasellus in nibh commodo, tempor magna in, convallis metus. Vivamus velit risus, scelerisque quis dolor in, finibus rhoncus erat. Vivamus ipsum libero, tempus non magna eget, condimentum tempus elit.
Sed eu feugiat neque. In velit ex, suscipit in semper blandit, malesuada in orci. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Donec sem odio, elementum at turpis in, aliquet posuere augue. Etiam accumsan libero lorem, tempor cursus purus fringilla in. Vestibulum id diam consectetur, interdum dui vitae, accumsan tellus. Ut eu viverra nisi. Duis odio nisl, consectetur id volutpat eu, interdum a tortor. In et ipsum interdum, fringilla urna nec, congue lectus. Aliquam eu sodales neque. Vivamus et tincidunt dolor. Sed porttitor rhoncus rutrum. Nulla facilisi.
Vivamus dapibus ipsum vitae libero ullamcorper, quis ullamcorper tortor porttitor. Phasellus elementum sapien ac felis sagittis, non finibus massa faucibus. Curabitur id enim neque. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Aenean velit lectus, tempus placerat pretium ultricies, mollis sit amet nibh. Praesent tincidunt turpis purus, vitae malesuada sapien eleifend at. Pellentesque velit mauris, volutpat auctor pharetra at, laoreet vel mi. Duis a ornare leo, nec malesuada ante. Donec a felis nec ex varius rutrum at a libero.
Etiam blandit nulla et lorem viverra, vitae suscipit mi luctus. Etiam enim nisl, dignissim eget lectus a, molestie hendrerit leo. Cras placerat leo nec blandit aliquet. Suspendisse id cursus metus. Aliquam a lobortis lectus, eget consequat erat. Praesent congue nulla vitae convallis pulvinar. Donec sed dui tellus. Aenean vehicula neque malesuada mi malesuada, sed lobortis nisl porttitor. Sed eu felis lacinia, fringilla nibh ac, laoreet ex. Vestibulum nibh ex, sagittis eu bibendum et, laoreet ut lectus. Proin ac augue tellus. Nulla tristique metus ut sem egestas sodales. In lorem sapien, tempor sit amet semper a, dignissim a dolor.
Mauris finibus justo ut pretium vestibulum. Morbi euismod faucibus fringilla. Curabitur vitae dictum ipsum. Curabitur nec nulla fringilla, laoreet ligula eu, convallis magna. Proin in accumsan sem. Morbi pretium venenatis sem, vitae fringilla leo vestibulum et. Maecenas justo ligula, iaculis a finibus nec, aliquam tempor ipsum. Donec cursus nisi vel purus pulvinar, non interdum nulla semper. In eu ullamcorper odio. Sed ac augue ut urna pulvinar rhoncus. Integer maximus ultrices nisl, nec volutpat tellus facilisis eu. Fusce dictum, leo iaculis egestas consectetur, enim ligula aliquam nunc, sed condimentum neque dui eget nibh. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae;
Fusce vitae orci eu purus vehicula viverra. Vivamus mollis orci sed euismod sagittis. Duis dui sapien, ullamcorper in gravida nec, imperdiet sed purus. Cras ligula nulla, consectetur a urna a, luctus ultricies augue. Aliquam tincidunt, lectus eget auctor venenatis, elit tortor malesuada mauris, sed iaculis lectus libero et lectus. Aenean dictum imperdiet tortor, ac aliquet magna rhoncus sed. Mauris facilisis velit suscipit ligula tristique ullamcorper. Praesent leo mauris, rhoncus eu sodales a, lobortis nec nibh.
Cras in libero felis. Donec luctus nunc id imperdiet consectetur. Nam ultrices suscipit mi, eu pretium urna luctus eget. Phasellus eu lacinia augue. Proin eu est condimentum ligula volutpat semper. Sed luctus, dolor quis bibendum venenatis, neque nibh condimentum felis, vitae cursus libero velit vitae lorem. Donec ultricies ullamcorper ipsum. Maecenas maximus accumsan blandit.
Mauris aliquet, ex non facilisis tristique, nibh elit efficitur quam, et gravida sapien leo sed diam. Suspendisse malesuada odio vel lorem dignissim, eu accumsan ante egestas. Vivamus blandit erat sed fringilla euismod. Etiam nec mauris a sem finibus dapibus. Quisque hendrerit eros nec mattis ultricies. Vestibulum blandit nulla a eleifend sollicitudin. Fusce hendrerit, nunc ut cursus fermentum, arcu odio laoreet turpis, a tincidunt purus massa nec sem. Nam id tellus et eros vehicula fermentum. Nullam imperdiet rhoncus lectus, at vestibulum nunc semper luctus. Sed a massa sed urna posuere congue in sed augue.
Nullam condimentum eget tortor in lobortis. Maecenas ac cursus tellus. Nunc mollis lorem risus, sed tincidunt sapien ullamcorper quis. In nec diam quis ligula euismod feugiat vitae eget dui. In pulvinar, arcu in molestie sodales, augue elit aliquam elit, vel dignissim quam mi maximus quam. Sed condimentum, nibh ut finibus faucibus, diam leo ultrices dolor, quis cursus nunc dolor non urna. Aliquam suscipit, magna vitae gravida porta, sem orci mattis arcu, nec fringilla dolor nunc in purus. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Nunc egestas cursus magna in ultrices. Maecenas quis laoreet ex, eu vehicula metus. Donec sed congue sem, in vulputate diam. Pellentesque molestie nulla ipsum, nec dignissim enim ultricies eget. Morbi vehicula odio ut justo tempus blandit. Sed nec condimentum elit. Morbi ut facilisis mauris. Aliquam luctus odio sed ante aliquam, eget venenatis risus luctus.
Integer laoreet odio a tellus tincidunt auctor sed id dolor. Praesent quis velit quis nunc dignissim iaculis non non lectus. Praesent blandit ligula urna, semper molestie lectus dignissim sed. Suspendisse bibendum, leo sed placerat tincidunt, sapien dui molestie dui, elementum dignissim nisl nisi et nulla. Ut feugiat felis id malesuada hendrerit. Pellentesque ut nisi et ipsum laoreet tempus vel non eros. Cras ut ante mi. Fusce sed maximus lacus. Etiam hendrerit, odio in maximus tincidunt, felis dolor malesuada justo, quis porttitor odio ipsum vitae eros. Vestibulum risus ante, iaculis sodales accumsan eget, tempor quis neque.
Vestibulum eget elit vestibulum, imperdiet ex ut, cursus metus. Proin at interdum leo. Vivamus a nisl tristique, varius nisl dignissim, auctor leo. Donec arcu felis, condimentum vel pharetra vitae, fringilla at dolor. Integer elementum viverra tortor, a ullamcorper nunc bibendum in. Vivamus et arcu sit amet nulla maximus condimentum. Vestibulum in nisi ut nulla sollicitudin gravida. Aliquam nulla ipsum, venenatis eu fermentum id, sodales vel diam. Suspendisse metus mi, facilisis ornare est et, interdum pretium odio. Morbi eget nunc orci.
Mauris neque dolor, imperdiet non dolor ut, suscipit lacinia mi. Donec dolor mauris, viverra in purus aliquet, tincidunt volutpat mi. Proin at dapibus dolor, vel egestas eros. Nulla mattis dictum iaculis. In pulvinar dui sem, eu tincidunt ligula sodales eget. Proin consectetur augue a libero suscipit rutrum blandit id eros. Pellentesque lorem erat, porta at felis id, congue malesuada urna. Quisque fringilla ut odio sed porta. Quisque congue lorem nec augue luctus varius. Nullam nec metus fermentum lacus egestas pharetra a volutpat lectus. Fusce euismod eros sit amet nisi semper imperdiet. Donec a viverra libero, vel ultrices felis. Aliquam vitae ante quis elit posuere ultricies. Mauris velit purus, tincidunt sit amet velit sit amet, sollicitudin pharetra odio.
Donec semper eleifend aliquet. Vestibulum fringilla augue non arcu tristique pellentesque. Duis viverra, eros vitae dignissim lobortis, mauris lorem ultricies tellus, non cursus diam tellus vitae ipsum. Ut et arcu turpis. Fusce eget neque cursus, posuere augue interdum, fringilla libero. Donec commodo velit finibus urna pellentesque blandit at eu turpis. Proin et viverra tellus, a pharetra sapien. Ut a odio fringilla, viverra elit in, dictum tortor. Morbi est diam, sagittis sed pulvinar sit amet, dictum at lorem. Phasellus a condimentum massa, sit amet vestibulum purus. Suspendisse quis pharetra tortor. Nunc tempus magna vitae ligula luctus laoreet. Integer eleifend varius commodo. In hac habitasse platea dictumst. Cras eget metus sapien. Nulla facilisi.
Cras euismod mauris tortor, a dapibus ligula gravida fermentum. Duis ultricies fermentum faucibus. Sed interdum, lacus vel mollis tempus, enim tellus ultrices nisi, in sollicitudin enim purus non nulla. Sed eget quam massa. In hac habitasse platea dictumst. Aenean at ante metus. Sed eleifend luctus ipsum nec lacinia. Vestibulum facilisis sodales dui, nec molestie neque tempus in. Curabitur consectetur tortor eget ipsum eleifend varius. Aenean finibus nulla at velit luctus, sed finibus ipsum semper. Vivamus turpis nisi, vulputate in pellentesque ultrices, rhoncus id augue. Quisque efficitur semper ligula, sed dictum turpis porta vitae. Aliquam malesuada est ac leo fermentum, et porttitor erat sagittis.
Morbi felis odio, tristique quis tempor at, convallis commodo lectus. Integer tincidunt lacus dolor, id molestie ante luctus non. Fusce nec quam in quam euismod malesuada. In consectetur magna ut fermentum volutpat. Phasellus malesuada risus nunc, non pellentesque mauris aliquet quis. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Praesent eget mi metus. Nunc in risus eget lacus gravida tristique a in nisi. Cras consequat aliquam quam vitae pulvinar. Curabitur commodo purus ligula, et ornare ipsum aliquet at. Sed tempor sed enim ut convallis. Mauris cursus magna non diam dapibus euismod. Nullam ac nisl est. Maecenas aliquet quam erat, ac imperdiet elit fermentum at.
Sed urna arcu, convallis et malesuada sit amet, iaculis quis felis. Fusce pellentesque tincidunt lacus, quis aliquam enim dictum vitae. Suspendisse potenti. Donec ut tincidunt est, eget iaculis leo. Curabitur auctor pharetra augue, sed egestas ante varius id. Etiam sollicitudin et mauris vitae ullamcorper. Maecenas mollis vulputate viverra. Etiam efficitur, metus quis cursus elementum, felis arcu congue dui, et volutpat augue tellus a dolor. Duis rhoncus molestie tincidunt. Nunc finibus tortor ut nunc vehicula, ac vestibulum velit tristique. Donec in eros ut erat tempor tincidunt.
Pellentesque cursus leo non nisl posuere, ac tincidunt lorem tempus. Praesent ut erat dictum, tincidunt elit ut, varius risus. Sed hendrerit id elit ut vestibulum. Suspendisse consequat metus sit amet neque dictum, sed feugiat risus egestas. Aliquam lobortis nisl elit, eget posuere ligula aliquam eget. Nullam lobortis a nunc vel malesuada. Praesent venenatis nisl sit amet libero suscipit, ut placerat sapien egestas. Cras condimentum justo sit amet massa sollicitudin, ac ultricies metus dignissim. Morbi mauris nunc, varius a ornare sit amet, pretium ut ex. Etiam sollicitudin, risus ut viverra euismod, magna mauris mattis tortor, eget cursus massa odio eu ipsum. Mauris tempus nunc mattis lectus varius cursus. Curabitur nisi erat, vulputate rutrum scelerisque vitae, convallis non lorem. Suspendisse purus nulla, aliquet eget hendrerit dignissim, malesuada nec orci.
In sagittis elit id augue iaculis euismod. Maecenas consequat odio sit amet massa elementum, eget fermentum velit varius. Aliquam ac tellus ac ex ullamcorper tincidunt eget eget diam. Quisque diam tortor, vehicula ac sollicitudin vitae, sollicitudin efficitur ligula. Nullam ut rutrum quam. Phasellus ornare posuere felis, sed vehicula ipsum blandit quis. Etiam a purus eu tortor interdum rutrum. Quisque sed tincidunt magna. Morbi sodales mi vitae sem cursus, sed venenatis augue porttitor. Nam posuere enim dictum hendrerit bibendum. Ut facilisis, dolor sed vestibulum ornare, tellus elit suscipit leo, et euismod arcu neque at tortor. Suspendisse pulvinar neque vel porttitor vestibulum.
Suspendisse in metus ut nibh euismod sodales. Sed tempor eget dolor at semper. Suspendisse at urna lacus. Donec quis velit sed elit ultricies vestibulum quis nec ipsum. Duis at augue et turpis gravida rhoncus quis in est. Fusce sit amet malesuada quam. Integer nec augue non nisl consequat scelerisque eu et velit. Sed vitae enim felis.
In a sem accumsan, iaculis nulla vitae, ultricies turpis. Nulla luctus, ligula a gravida dapibus, mauris mauris rutrum erat, et lobortis libero libero sed nibh. In quam diam, dapibus vitae diam in, interdum accumsan ligula. Phasellus ac diam mollis, laoreet sapien ut, vehicula quam. Donec cursus elit tortor, vel mattis odio ornare ut. Quisque et justo a purus aliquam laoreet. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Nulla non euismod metus. Integer venenatis eu nisl tempus consectetur.
Phasellus dictum elit vel velit rhoncus, porttitor tempor mauris scelerisque. Quisque nec fringilla erat. Sed consectetur in eros ac maximus. In nec lorem sapien. Pellentesque aliquet bibendum mi, at pulvinar justo mattis nec. Proin justo lorem, tempus nec elit lobortis, interdum pretium nisl. Pellentesque euismod, massa a consectetur dignissim, risus purus dictum risus, in molestie dolor elit in turpis. Cras vitae dapibus augue.
Proin enim diam, semper ac dapibus eget, vulputate id ligula. Proin lectus diam, pharetra sed turpis non, varius pharetra eros. Quisque eget rhoncus enim. Integer velit ante, molestie eget convallis vitae, laoreet eget massa. Etiam at sem nec urna accumsan convallis. Nam a diam luctus, scelerisque nisi id, pulvinar quam. Aliquam convallis maximus aliquet. Aliquam at diam nec tellus pretium euismod. Cras aliquam justo nec quam scelerisque vulputate. Etiam dictum eleifend elit elementum consequat. Donec semper tempus ultrices. Pellentesque bibendum vitae dolor vel scelerisque. Aenean lacinia hendrerit dolor non congue.
Ut congue orci turpis, sit amet ultricies orci luctus in. Ut felis odio, vestibulum non convallis sit amet, congue vitae mauris. Nullam blandit enim vel lorem laoreet, at gravida est sollicitudin. Aenean posuere dignissim ex, id varius arcu iaculis id. Vestibulum id nulla eget magna pulvinar rutrum. Suspendisse pulvinar blandit mauris, vel pharetra turpis finibus a. Quisque ac ligula arcu. Praesent semper nulla sed ultrices scelerisque. Quisque id erat eget odio dictum euismod. Donec sit amet nunc purus. Quisque nulla dui, sollicitudin non odio sit amet, sagittis interdum urna. Nunc feugiat, lacus non commodo volutpat, tellus lorem fermentum risus, eget dapibus urna massa a elit.
Sed id tellus augue. Donec quis fringilla lacus. Integer suscipit faucibus eleifend. Donec lobortis odio ut felis cursus rutrum. Morbi augue erat, rutrum eu nisl sed, tincidunt porta enim. Nulla consequat malesuada tellus. Pellentesque facilisis vel nibh et pretium. Morbi volutpat ante sed leo tincidunt, egestas bibendum dui auctor. Morbi mattis feugiat maximus. Donec a sagittis ante, non euismod metus. Morbi commodo neque viverra pretium fermentum.
In sodales, nisl quis vulputate luctus, sapien est fringilla elit, sed vestibulum urna libero ac ante. Suspendisse potenti. Duis eget sagittis elit. Mauris sapien ligula, egestas at auctor eu, efficitur at nisi. Proin elementum, erat nec tincidunt laoreet, elit risus pellentesque sapien, in malesuada enim ligula id magna. Proin scelerisque augue lorem, et hendrerit ante fringilla vel. Quisque in faucibus nunc, sit amet convallis diam. Sed fermentum tristique fringilla. In condimentum purus ornare tristique dapibus. In malesuada nunc lorem, vel imperdiet erat pellentesque id. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Quisque feugiat faucibus nulla, quis faucibus velit lobortis in. Donec augue sem, scelerisque vitae tortor ac, aliquet fermentum nulla. Fusce convallis non metus in ultrices. Cras justo arcu, tristique vel libero sed, fermentum ullamcorper justo. Mauris libero erat, elementum nec malesuada ac, commodo eget ante.
Duis laoreet diam non orci volutpat rhoncus. Sed bibendum dolor quam, eget sagittis enim tincidunt at. Mauris at varius sem, id luctus augue. Sed venenatis pulvinar viverra. Curabitur enim nisi, mollis at fermentum ac, rhoncus iaculis mi. Ut dictum urna velit, a rhoncus risus tempus ut. Cras tristique scelerisque dignissim. Donec ex felis, dictum at eleifend at, posuere bibendum quam. Donec luctus aliquet velit, id fringilla sem tincidunt sed. Quisque cursus imperdiet diam, ut facilisis augue convallis et. Aliquam hendrerit consectetur neque, vitae ultricies nulla aliquam ut. Donec at justo ut ipsum aliquam bibendum in id ante. Aenean fermentum eros vel turpis tristique egestas. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Praesent vitae dui felis.
Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Nam faucibus leo id mi blandit, posuere maximus felis malesuada. Phasellus et porttitor magna. Curabitur leo ipsum, malesuada at lobortis finibus, facilisis id purus. Suspendisse potenti. Suspendisse at iaculis metus. Nam pharetra leo quis ex aliquam fermentum. Sed quis metus faucibus, varius nunc id, condimentum est. Nam lacinia quis velit a iaculis. Nullam accumsan mattis neque vitae posuere. Vivamus sem neque, ultrices sed molestie at, gravida ut est. Nunc a tellus viverra felis pulvinar fermentum vitae nec mi. Nulla et hendrerit magna, sed bibendum mauris. Cras eget diam eu augue convallis porttitor eget sit amet tortor. Cras arcu tortor, vulputate vitae erat non, rutrum rhoncus urna. Donec blandit non erat sit amet gravida.
Sed feugiat in nibh et sagittis. Quisque in maximus mi, eu elementum neque. In hac habitasse platea dictumst. Pellentesque ultricies consectetur urna vitae imperdiet. Nullam velit lectus, laoreet ut sem eu, commodo fringilla ipsum. Vivamus placerat vulputate ipsum nec viverra. Aenean vel venenatis augue, vitae pharetra felis.
Pellentesque rutrum urna orci, a condimentum mi ultrices quis. Nunc facilisis velit nec velit eleifend vestibulum et vel erat. Fusce consequat ex ut lacus elementum lacinia. Nulla a sapien ut ex dignissim pulvinar sed vel ex. Aenean porta diam sit amet pellentesque dignissim. Vestibulum mollis convallis auctor. Etiam lacinia eros non nulla blandit tristique. In hac habitasse platea dictumst. Vestibulum dapibus iaculis consectetur. Morbi ex odio, posuere at sollicitudin mattis, efficitur pharetra sapien. Etiam placerat nec quam vitae fringilla. Donec sodales bibendum odio, eget pharetra erat efficitur id. Nullam ultricies dui odio, sit amet tincidunt eros vestibulum eu. Donec semper libero in lacus elementum maximus.
Curabitur commodo ex nec sapien fermentum suscipit. Donec vel erat placerat, convallis dui non, mattis mauris. Donec placerat dui augue, et dignissim justo feugiat id. Phasellus nec justo ex. Phasellus eget lobortis orci. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Cras in scelerisque neque, non pretium purus. Aenean rutrum libero at fermentum pharetra. Nam elementum felis nec dapibus dignissim. Quisque ultricies ipsum a odio pellentesque mollis a maximus leo.
Sed id urna hendrerit, convallis nisl quis, tristique felis. Sed eget consequat nisl. In vel tincidunt erat. Vivamus molestie rhoncus libero, non tincidunt lacus elementum non. Aenean faucibus lacinia nisi ac lobortis. Suspendisse iaculis augue nibh, id varius tortor ornare ac. Duis vitae congue nibh. Praesent at tortor et justo aliquam fringilla vitae id orci. Mauris ultricies velit condimentum lectus volutpat, vitae tincidunt odio fermentum. Curabitur luctus convallis libero eget pellentesque. Vivamus a tempus ipsum.
In et justo vel nisi convallis tempor ac vitae purus. Cras efficitur ac orci sit amet aliquet. Vestibulum vel libero egestas, pharetra nisl a, ultrices erat. Nunc id turpis a erat aliquet hendrerit. Suspendisse lobortis est sed quam consectetur condimentum. Proin fermentum purus nec risus dignissim lacinia. Quisque eu libero eu nibh dictum congue id at odio. Nulla posuere justo a mollis scelerisque. Nunc luctus, augue congue volutpat tincidunt, orci nulla euismod elit, eget mollis arcu augue eget elit. Aliquam rhoncus nisl at quam dictum viverra.
Pellentesque sit amet elit condimentum, suscipit sapien sed, dapibus turpis. Proin felis nunc, aliquet in vulputate a, lobortis et ex. Donec ac magna vulputate, tincidunt mauris eget, ultrices urna. Duis venenatis commodo massa, eget rutrum enim consectetur vel. Aenean vel erat hendrerit, tincidunt dui ut, elementum est. Vestibulum in ipsum fringilla, gravida nunc ac, sagittis dolor. Aenean pulvinar ornare diam eget ultricies.
Cras ut luctus mauris, sed sodales orci. Quisque vitae ullamcorper metus. Ut vel justo ligula. Aenean sit amet tellus tortor. Mauris eu diam et mauris vestibulum vehicula. Donec finibus, turpis vel blandit pretium, sem quam sagittis purus, in sagittis nibh leo id augue. Duis venenatis mollis pretium. Praesent pretium bibendum eros. In non ipsum cursus, tristique orci vel, elementum dolor. Nulla egestas leo in feugiat dignissim. Integer fringilla odio ut aliquam accumsan.
Sed risus est, tristique ut ex quis, aliquam malesuada lorem. Maecenas hendrerit eros ultricies venenatis aliquam. Vestibulum ut laoreet lectus. Integer purus neque, porttitor sed tristique congue, vestibulum et ligula. Aliquam fringilla, eros et mattis vulputate, tellus urna auctor velit, ac pharetra ligula mauris ut tortor. Donec tristique nunc metus, vitae vulputate nulla iaculis vitae. Donec iaculis dapibus dolor, eget rhoncus dui. Ut feugiat sed enim tristique efficitur. Curabitur leo risus, vehicula ac ligula id, vestibulum eleifend diam. Aliquam erat volutpat. In ipsum diam, volutpat at diam non, finibus lobortis eros. Curabitur id diam mi. Proin purus urna, auctor et diam nec, aliquam interdum lectus. Ut eget mollis tortor. Quisque elementum porta ultrices.
Nullam aliquet augue velit, sed suscipit erat eleifend non. Vivamus nisl felis, blandit sit amet neque id, malesuada tincidunt mi. Phasellus a mauris metus. Cras tempor, arcu tincidunt fermentum viverra, tortor lacus tincidunt erat, vel tristique dolor justo vitae eros. Aliquam erat volutpat. Cras aliquet nunc et dignissim sagittis. Fusce vel nisi mi.
Nunc tempus purus non magna tincidunt, eget dignissim justo posuere. Aenean mattis lacinia risus vel luctus. Suspendisse ac rhoncus massa, id finibus dui. Pellentesque nulla turpis, iaculis vitae mauris non, hendrerit tempus erat. Integer venenatis, dui in rutrum porttitor, purus risus commodo nisl, a fermentum nisi nunc eu neque. Quisque euismod est nec mi facilisis, ut varius leo congue. Integer sed arcu ultrices, volutpat diam at, elementum turpis. Quisque et accumsan orci. Duis consequat sollicitudin tortor in ultrices. Vestibulum porta fringilla auctor. Maecenas maximus eros at erat vestibulum mattis. Praesent fringilla pellentesque quam, vel ullamcorper nisi sollicitudin non. Curabitur fermentum fermentum ligula sed viverra.
In hac habitasse platea dictumst. Quisque et convallis est, quis posuere felis. In congue, elit nec venenatis hendrerit, eros sapien dictum erat, non vehicula nibh felis ac sem. Nam sed semper massa. Proin id accumsan lorem. Mauris ultrices leo et velit euismod facilisis. Nulla facilisi. Morbi ultrices, mauris id ullamcorper sodales, ex neque eleifend tellus, sed luctus neque orci a dui. Curabitur ut eros metus. Morbi rhoncus odio eget lacinia blandit. Aenean lobortis consequat imperdiet. Proin tempus vehicula massa, nec posuere ex. Phasellus convallis, felis ac lacinia luctus, purus nunc imperdiet ante, eu hendrerit nibh diam eu lacus.
Vivamus ullamcorper molestie turpis, et euismod lorem semper ac. Proin ornare, purus at ullamcorper euismod, lacus odio gravida nisl, vitae pretium mi erat a sapien. Duis ultrices libero turpis, sit amet varius sapien tempus in. Integer eget dignissim est. Aenean eget nulla nec libero faucibus tempus. Etiam in pellentesque risus. Nunc sed luctus lacus. Duis tristique nulla non enim consequat, congue vestibulum nisl interdum. Proin faucibus, eros non accumsan rutrum, ipsum justo fermentum augue, tempor ornare est metus sit amet tellus. Duis malesuada vel justo at finibus. Nullam sit amet enim scelerisque, dapibus velit ut, iaculis lectus. Nunc elementum erat nibh, eget finibus dolor porta in. Fusce varius tellus mattis tellus commodo pellentesque. Cras viverra gravida ligula, quis hendrerit ex posuere vitae. Sed quis tempor felis, ultrices faucibus velit.
Quisque porttitor mi vitae metus dapibus, eu tincidunt turpis pharetra. Fusce dolor nulla, vulputate a ligula sed, suscipit hendrerit sem. Integer id nunc vitae erat dictum tempor. Morbi leo dui, tincidunt sed dapibus non, vestibulum sit amet magna. Proin et mauris tellus. Nam volutpat orci eu eros imperdiet congue. Suspendisse nec dictum magna. Nunc consectetur varius augue a ultricies. Proin nec lacus eget massa mattis ornare eget id ligula. Sed laoreet ante nec efficitur lacinia.
Integer maximus laoreet tellus, eget aliquam mauris luctus ut. Sed condimentum lectus et mi eleifend egestas. Integer convallis tempus sem laoreet consequat. Suspendisse gravida commodo purus eu consequat. Pellentesque sed mi efficitur, tristique felis et, tempus lacus. Nullam in dictum est. Aenean libero libero, ullamcorper a cursus eget, mattis vitae diam. Aliquam at dolor turpis.
Ut mauris justo, accumsan molestie eros vitae, interdum ornare lorem. Mauris ut consectetur nulla, cursus vulputate lectus. Donec commodo urna velit, nec lacinia elit accumsan sit amet. In et augue vel leo rutrum vulputate at vel magna. Phasellus in tempus erat, quis elementum mauris. Vestibulum tincidunt facilisis ante, ut volutpat eros rhoncus vitae. Aliquam erat volutpat. Proin sit amet lacus turpis. Etiam maximus sodales libero, ut vestibulum dui lacinia sit amet. Interdum et malesuada fames ac ante ipsum primis in faucibus. Donec scelerisque, turpis et auctor dictum, dolor odio sodales ligula, a convallis dolor erat a augue. Sed felis nisi, tristique sit amet dolor sit amet, imperdiet auctor mauris. Aenean vitae ex molestie, suscipit lorem fermentum, laoreet nisi.
Suspendisse pellentesque facilisis erat, sed faucibus sapien facilisis et. Proin pharetra augue ut lorem viverra pulvinar. Fusce quam neque, egestas sed est non, rutrum porta justo. Duis fermentum tincidunt ipsum non pharetra. Suspendisse potenti. Ut volutpat magna quis erat vehicula posuere. Aliquam consequat consequat gravida. Etiam dictum gravida semper. In vitae maximus felis. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas.
Aenean pretium, nisl ac dapibus lobortis, sem justo fermentum magna, in ultricies ipsum felis vitae ex. Pellentesque dictum eget purus sit amet commodo. Nunc dapibus, leo eget bibendum luctus, leo nibh eleifend lectus, eget imperdiet felis sem id nunc. Nullam egestas justo commodo, blandit ante at, placerat dolor. Nunc malesuada nulla et nisi maximus, quis molestie diam blandit. Quisque elementum lacus purus, quis varius lectus sodales finibus. Donec sodales nunc non nunc tincidunt, eu rutrum arcu pretium. Etiam risus odio, consequat eget quam eu, dignissim iaculis erat. Aliquam eu porttitor urna. Donec molestie, diam quis tempus maximus, leo mauris pretium mauris, ut lobortis est ipsum a mauris. Nam dignissim congue leo, id mattis sem efficitur eget. Proin risus lorem, fringilla vel varius ac, hendrerit vel neque. Vestibulum auctor est fermentum, congue purus at, semper dolor. Maecenas nec nunc at dolor blandit suscipit. Donec eu nisi aliquam, eleifend sem a, ultrices ex.
In at augue risus. Nam sed justo in quam porta maximus. Praesent elementum tellus sed dui gravida hendrerit. Cras ultricies pulvinar lobortis. Maecenas tortor augue, auctor ut eleifend eget, egestas at lectus. Aliquam erat volutpat. In in egestas tellus.
Quisque volutpat placerat vulputate. Nulla aliquam consectetur ex a vulputate. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla volutpat leo elit, id blandit erat vehicula eget. Pellentesque molestie, lacus ac facilisis fermentum, turpis enim faucibus felis, sit amet rhoncus libero dui at mauris. Proin vel placerat risus. Nullam eleifend orci eget eros tempor, aliquet semper diam malesuada.
In convallis gravida laoreet. Praesent scelerisque mollis massa, non lobortis sapien elementum id. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Etiam dignissim varius eros vitae molestie. Aenean molestie nisl ac bibendum laoreet. Sed placerat pellentesque augue, sit amet commodo turpis condimentum vitae. Duis a purus placerat, tempor nulla ac, rutrum ipsum. Donec lectus nisi, laoreet quis diam eget, finibus scelerisque ante. Aliquam erat volutpat. Integer convallis justo vel nibh vehicula molestie. Donec ut mi eget est pulvinar euismod. In id ligula ut enim venenatis fermentum. In metus leo, rutrum ac mollis tincidunt, semper quis lacus. Sed lobortis, augue in condimentum suscipit, augue velit tempor leo, eu mollis turpis lectus auctor orci.
Nullam efficitur erat ut dolor tristique condimentum. Suspendisse pretium ex ut bibendum ullamcorper. Phasellus dapibus enim sed tellus condimentum, sit amet mattis tellus cursus. Phasellus venenatis augue in lacus suscipit, vel hendrerit felis vehicula. Cras consectetur cursus lorem vulputate cursus. Vestibulum nec auctor sem. Nam ultricies, mi a ullamcorper eleifend, enim magna elementum erat, a accumsan turpis nunc et tortor. Donec venenatis maximus lacus vitae sodales. Donec dapibus rutrum porta. Nulla facilisi. Vivamus aliquam ex vitae sem consequat blandit. Nullam ultricies, ante ac rutrum efficitur, magna ligula maximus turpis, ut porta est velit et magna. Vestibulum mauris massa, posuere nec convallis in, maximus a nulla. Mauris vitae lorem sed nulla aliquam tempor. Fusce vel fringilla metus, ac aliquet metus.
Ut ac mattis augue. Fusce cursus at quam ut vehicula. In laoreet cursus urna eu fermentum. Suspendisse est mauris, gravida interdum urna a, ullamcorper hendrerit arcu. Donec dapibus blandit massa nec vulputate. Nam a lacus pretium, imperdiet risus ac, aliquam nunc. Nulla facilisi. Vivamus quam libero, vestibulum sed tellus eget, ornare gravida dolor. Sed placerat nulla in velit imperdiet mattis. Sed at arcu eleifend, scelerisque urna non, porta massa. In volutpat commodo quam et sollicitudin.
Donec in magna ullamcorper, auctor ex malesuada, tincidunt dui. Etiam enim risus, cursus sit amet ante ut, blandit tincidunt purus. Etiam rutrum dolor nulla, vel feugiat quam convallis sit amet. In finibus mi tortor, non hendrerit purus dictum at. Vivamus condimentum elementum neque et maximus. Cras auctor iaculis metus, at vulputate justo rhoncus eu. Aliquam laoreet mi euismod, lacinia est nec, euismod augue. In viverra tincidunt dolor vitae porttitor. Proin congue, mi eu laoreet congue, libero nunc porttitor tellus, non dictum magna erat sed purus. In nec luctus ligula, vel gravida urna. Sed lacinia mollis justo at hendrerit. Pellentesque gravida laoreet risus non auctor. Praesent ac sollicitudin eros. Vestibulum non viverra magna, sodales tristique ipsum.
Etiam et lacinia eros, ut scelerisque turpis. Sed elit tortor, varius in nibh at, tempor euismod massa. Sed dapibus purus nec felis venenatis, nec rhoncus eros sagittis. Nullam elit orci, facilisis nec nunc sed, lobortis sagittis metus. Proin aliquam pharetra sagittis. Etiam ultrices nulla quis posuere elementum. Sed ultrices justo justo, eu tempor turpis rutrum vel. Pellentesque at cursus tellus. Vestibulum pulvinar tellus eget felis posuere bibendum. Etiam nec orci eleifend, gravida ipsum mollis, facilisis erat. In efficitur ac metus vel aliquam. Quisque arcu est, malesuada ut ligula ac, consectetur rutrum ex. Quisque varius viverra gravida. Suspendisse id leo quis felis imperdiet fringilla. Aenean ac accumsan urna. Fusce sapien mauris, varius pellentesque porta lobortis, tempus scelerisque metus.
Vivamus tempus interdum felis, quis gravida ipsum auctor at. Cras sed ligula eu mauris semper pellentesque. Donec ut nibh at odio tempus euismod. Interdum et malesuada fames ac ante ipsum primis in faucibus. Aliquam ultrices odio ut faucibus mollis. Curabitur tincidunt accumsan luctus. Suspendisse tincidunt magna mi, et euismod nulla feugiat quis. Fusce finibus ac velit id fermentum. Aliquam venenatis egestas aliquam. Nulla diam libero, consectetur eu enim ac, dictum tempor lectus. Ut id tempus augue. Nulla facilisis, massa sit amet ultricies ultrices, eros lacus eleifend ex, vitae facilisis velit urna non leo. Ut libero ligula, venenatis ac odio id, posuere hendrerit eros. Cras eget sapien at mauris iaculis tincidunt. Nullam ut neque nisi. Aliquam hendrerit, magna non pellentesque iaculis, nulla libero molestie augue, non vehicula tortor sapien porttitor eros.
Sed eget lectus nec enim porta gravida. Vestibulum id tincidunt nunc. Quisque scelerisque condimentum ipsum, eu accumsan orci facilisis non. Duis venenatis, est et mattis finibus, turpis urna rhoncus urna, gravida ultrices ipsum neque vitae erat. Vivamus massa enim, tristique vulputate faucibus a, luctus non dui. Vestibulum non risus lorem. Suspendisse eu orci accumsan, molestie enim nec, convallis augue.
Nullam sed arcu turpis. Quisque ut elementum velit. Maecenas vitae sem vel eros vehicula hendrerit. Aenean suscipit convallis justo. Morbi in consectetur diam. Donec et efficitur justo, vestibulum convallis turpis. Proin sit amet enim id enim tempor tempus. Nam est augue, consectetur vitae ligula ut, tristique consequat nibh.
Ut pharetra tortor auctor risus posuere, nec hendrerit nibh rhoncus. Aliquam tincidunt aliquam felis, at dignissim justo fermentum quis. Aenean malesuada, eros a ornare varius, sapien ex mollis nunc, vitae bibendum augue lectus vel mi. Nulla ultricies dui sed tellus sodales, at iaculis urna elementum. Sed et lacinia urna. Integer commodo nulla non quam rhoncus, bibendum tristique massa finibus. In posuere, lectus id tristique tristique, risus mauris consectetur erat, fringilla mattis metus eros in velit. Praesent est eros, tempor vel vulputate ut, luctus ut libero. Phasellus a urna porttitor, aliquam enim non, varius ipsum. Cras non ante ut ante egestas condimentum. Praesent finibus eleifend eros, tempor feugiat neque semper eget. Duis ullamcorper condimentum aliquet. Interdum et malesuada fames ac ante ipsum primis in faucibus.
Curabitur eget orci aliquet, pulvinar enim quis, hendrerit lectus. Suspendisse sed dictum lorem, nec porttitor ligula. Aliquam sit amet ligula sed lorem consectetur rhoncus ut a mauris. Quisque a ipsum sit amet augue mollis lobortis id nec risus. Phasellus vulputate justo in eros sodales vulputate. Fusce leo magna, condimentum quis vehicula id, malesuada at justo. Donec fringilla tortor in ullamcorper viverra. Vestibulum efficitur, quam ac consectetur dignissim, metus libero tincidunt libero, quis ultrices nisi mi eu erat. Nulla pharetra iaculis ullamcorper. Donec sit amet tortor congue risus elementum venenatis. Maecenas nisl nunc, imperdiet nec mattis sit amet, dignissim eget sapien. Nullam tristique turpis eu ante tempor, nec suscipit tortor sodales. Quisque cursus a orci quis molestie. Sed sit amet venenatis leo. Maecenas felis lacus, accumsan at accumsan non, ornare quis dolor.
Aliquam vel enim eros. Curabitur sit amet risus ligula. Aliquam eget iaculis lacus, vitae efficitur nisl. Vestibulum convallis, risus at lacinia tempus, libero tortor rhoncus augue, id fringilla ipsum massa eget lorem. Maecenas justo leo, dignissim vitae finibus a, vehicula quis eros. Vivamus vel nisl porttitor, aliquam sapien et, semper risus. Praesent at sem tellus. Fusce sit amet fringilla elit. Nullam lorem sapien, vulputate eget lacus interdum, fermentum rutrum neque. Nulla scelerisque massa a felis cursus euismod.
Mauris a mi posuere, eleifend velit in, luctus nisi. Donec mattis lacus velit, non laoreet odio posuere ac. Nulla efficitur fringilla orci a porta. Vestibulum est magna, fermentum id tempor eget, volutpat vel magna. Nunc non aliquam urna, ut congue urna. Aliquam purus nunc, pretium eget vehicula sed, vehicula sed sem. Quisque pellentesque velit sit amet orci dapibus tristique. Cras fringilla velit id ultrices scelerisque. Praesent porta egestas mauris, vitae accumsan quam. Aliquam molestie, magna sit amet maximus feugiat, arcu mauris ultricies lorem, id aliquam turpis arcu ac mauris. Etiam eu scelerisque neque. Donec sed quam vel est dictum convallis quis posuere tortor. Sed sit amet tortor eros. Sed odio purus, egestas at lacinia sed, consectetur id diam. Quisque tincidunt, ante eget mattis cursus, felis ante venenatis leo, in eleifend erat diam eu eros.
Sed ac nunc mauris. Sed hendrerit ligula efficitur facilisis tincidunt. Morbi ut ornare lorem, sed facilisis ex. Aenean aliquam tristique mi, ac tristique metus rutrum eget. Curabitur id leo id massa commodo sagittis in at elit. Morbi viverra bibendum ligula vitae tristique. Etiam at est interdum, euismod nibh nec, condimentum arcu. Etiam orci ex, sagittis et tincidunt quis, finibus eu sapien. Sed urna lorem, suscipit a gravida vitae, sollicitudin vitae dolor. Cras ut imperdiet massa. Fusce ornare iaculis ipsum a cursus. Pellentesque vulputate, justo vel tincidunt egestas, lacus odio convallis odio, eu porttitor felis ipsum vitae libero.
Phasellus eget nulla eget est convallis ornare ac non enim. Integer tincidunt massa eu tincidunt euismod. Proin at nulla in dui malesuada venenatis vitae at ligula. Curabitur dapibus mauris vitae turpis euismod, et pharetra quam molestie. Nulla id faucibus tortor. Pellentesque ultrices, turpis vel lobortis fermentum, sapien diam rhoncus sapien, quis tristique turpis lorem a mi. Nam bibendum, sem eget congue interdum, lacus orci convallis elit, ac porttitor lorem erat et orci. Integer elementum tortor et nisl posuere consectetur. Sed malesuada leo ac urna lacinia, malesuada luctus mauris faucibus. Duis consequat posuere lobortis. Cras interdum lacinia lacus. Cras ipsum sapien, porttitor eget pellentesque ut, aliquam ut magna. Integer luctus velit et elementum malesuada. Maecenas tempus mauris quis mollis posuere.
Maecenas consequat urna elit, eget fringilla felis laoreet ac. In congue ullamcorper odio, sed malesuada turpis luctus sed. Nullam sodales interdum elit ac dignissim. Pellentesque placerat mollis velit, vulputate tristique neque aliquet vitae. Phasellus tempor viverra est, vitae vehicula justo imperdiet nec. Fusce dictum lacinia urna eget rhoncus. Nam imperdiet nisl orci, sed ultricies orci laoreet vel. In in pulvinar eros. Ut eu rhoncus eros. Suspendisse eu dui viverra, lacinia erat vitae, mattis metus. Aliquam vitae posuere sapien. Quisque eu lorem quis mi egestas varius. Aliquam erat volutpat.
Duis eu arcu lobortis, vehicula orci ac, imperdiet dui. Etiam venenatis nisi quam, quis dapibus erat sagittis non. Suspendisse lacinia blandit interdum. Vivamus vitae sollicitudin leo. Vivamus sit amet commodo tellus, ac sagittis augue. Vivamus mi orci, ultricies ac nulla at, pharetra maximus diam. Nullam rhoncus volutpat magna eu auctor.
Etiam commodo enim a leo pellentesque, in elementum odio lobortis. Vestibulum lobortis lobortis malesuada. Sed imperdiet ullamcorper viverra. Cras facilisis malesuada purus a consequat. Ut auctor neque mi, in scelerisque nibh ornare eu. In non dui in enim pretium ullamcorper non rutrum urna. Donec dictum porta orci sed malesuada. Morbi ac placerat felis. Aliquam lacus sem, ullamcorper sed laoreet ut, imperdiet non libero. Nunc non metus id justo accumsan ultricies. Donec in lacinia eros. Morbi non nunc diam. Aenean nisl massa, vestibulum vel fringilla vel, placerat eu leo. Sed feugiat malesuada ultricies.
Nulla tristique massa eu tortor feugiat auctor. In viverra eu ex quis auctor. Praesent ac sapien orci. Proin sit amet orci posuere, ultricies orci in, fermentum justo. Vivamus tempus, ligula et aliquam egestas, enim orci pulvinar dui, in ultricies mauris turpis eu nisi. Pellentesque nec dui in ipsum laoreet convallis. Vestibulum a mi ornare, elementum ex sed, rhoncus neque.
Nullam sed orci id sem tincidunt suscipit. Cras ac leo at magna facilisis blandit. Aliquam vitae tristique nisi. Nulla vestibulum felis pretium lectus commodo, id eleifend risus eleifend. Mauris bibendum facilisis est vitae scelerisque. Cras interdum dapibus ligula, nec porta lacus imperdiet at. Cras rhoncus bibendum lorem, congue mollis libero dignissim eget. Sed gravida, mauris sit amet sodales posuere, tellus felis imperdiet arcu, quis iaculis orci orci vitae ipsum. Curabitur id orci odio. Aliquam vitae rutrum lacus.
Maecenas tristique est felis, eget posuere lorem dapibus non. Maecenas eu interdum lectus. Nullam placerat sit amet quam non hendrerit. Nulla facilisis ornare mollis. Cras sed metus facilisis, mattis dui in, auctor est. Morbi vehicula venenatis est, vitae egestas felis tincidunt vitae. Donec iaculis massa id justo ornare rhoncus. Nulla tempor felis ex, eu consectetur justo dictum sed. Integer eget laoreet nibh. Duis vitae pellentesque tellus, id blandit justo. Cras quis mollis eros. Quisque rhoncus dignissim enim at sollicitudin. Vestibulum sit amet diam sed quam sodales finibus. Nulla maximus orci sit amet porttitor vestibulum. Nam consequat urna at varius vulputate.
Suspendisse laoreet luctus mauris. Aenean mollis felis urna, ac pretium nunc rutrum non. Aenean semper, risus vitae iaculis eleifend, odio ligula tempus nunc, sit amet suscipit nibh lorem quis ipsum. Nam et placerat sapien. In luctus accumsan risus, id pulvinar elit venenatis eu. Phasellus elementum leo urna, a dapibus neque facilisis eget. Sed sit amet tempor tortor. Fusce iaculis, ipsum nec faucibus scelerisque, felis tortor condimentum purus, in fringilla ex est ac nisl. Nunc et interdum diam. Donec venenatis, dolor quis dignissim euismod, libero ante elementum libero, vitae laoreet purus velit sit amet ipsum. Quisque urna tellus, imperdiet eget gravida a, fringilla commodo diam.
In posuere nisi dictum tortor elementum iaculis. In dignissim diam sit amet volutpat elementum. Curabitur ultricies mi libero, sit amet feugiat massa viverra sed. Vivamus justo nibh, commodo nec elementum eu, suscipit vel elit. Fusce ac cursus libero, et volutpat augue. In tempus ultricies libero, ac rutrum mauris. Morbi vestibulum tellus eu dui dignissim euismod. In hac habitasse platea dictumst.
Nam rhoncus hendrerit ex et mattis. Sed varius, arcu quis placerat viverra, ex lorem ultrices arcu, nec fringilla ipsum metus ut ligula. Sed in luctus mi. Pellentesque sed eros nisl. Praesent rutrum magna metus, vel efficitur eros lobortis efficitur. Mauris vestibulum urna at ligula lobortis sollicitudin. Aenean rhoncus auctor leo vel interdum. Cras sollicitudin massa leo, et eleifend metus scelerisque sit amet. Praesent dapibus euismod libero a fermentum.
Curabitur cursus lacus sit amet est feugiat euismod. Nulla accumsan risus congue lorem facilisis scelerisque. In sed lectus elementum, porttitor nisl vulputate, pulvinar mi. Maecenas eu mollis odio. In faucibus sagittis magna, vitae mollis nisi iaculis non. Aenean dictum lectus ac arcu vehicula fringilla. Curabitur accumsan efficitur libero, eget consequat magna ultrices vel. Donec fermentum vel orci eget finibus. Etiam in massa ante. Mauris gravida enim lacus, sit amet accumsan massa suscipit a. Mauris id bibendum ex, et convallis nisl. Morbi luctus, orci in malesuada finibus, neque turpis convallis justo, id gravida sem purus eget turpis. Fusce eu laoreet justo.
Nulla facilisi. Nulla varius risus quam, sit amet aliquam felis lacinia a. Ut sapien felis, tincidunt in pellentesque sit amet, vehicula id elit. Morbi a congue nunc. Sed justo neque, rutrum tincidunt hendrerit in, luctus at mauris. Suspendisse porttitor ex vitae felis luctus, et bibendum eros consequat. Maecenas vitae lectus eget est volutpat fermentum ac ac elit. In hac habitasse platea dictumst. Aenean luctus ex nec orci euismod aliquam. Integer a dolor elementum, mollis magna vitae, porta dui. In ac erat posuere, facilisis lacus eget, venenatis velit. Nulla ut sapien tincidunt, ultricies lectus aliquet, varius odio. Curabitur viverra congue ipsum, non finibus enim lobortis vitae.
Duis vestibulum maximus est, sollicitudin dapibus sapien accumsan sit amet. Cras luctus, massa malesuada ornare imperdiet, dolor lorem blandit est, a blandit erat quam vel tortor. Vestibulum id semper ipsum. Ut nec ante eget velit fringilla sagittis. Duis sit amet lobortis nisi. Aenean interdum dui ut metus suscipit, a pretium tortor ultrices. Nullam tincidunt bibendum nisl, vitae tincidunt urna tincidunt tempus. Donec vitae porta sem. Phasellus venenatis egestas ligula, quis volutpat ipsum lacinia et. Pellentesque placerat ipsum elit, a feugiat libero scelerisque fringilla. Suspendisse ullamcorper congue nisi ut fringilla. Aliquam quis suscipit orci. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Nam at nulla volutpat, viverra sem ac, blandit erat. Quisque gravida quam eu magna hendrerit, quis varius diam scelerisque. Suspendisse nisi enim, dignissim sit amet tristique ut, tempus sodales nibh.
Nulla ac eleifend leo. Quisque laoreet finibus mattis. Nam est nulla, maximus sed luctus non, volutpat quis ipsum. Praesent et quam dolor. Nulla facilisi. Aliquam enim orci, volutpat vitae mattis pellentesque, fringilla ut eros. Fusce semper arcu et sapien rutrum laoreet. Maecenas vel imperdiet lectus. Mauris et pulvinar justo. Mauris sed luctus diam. Fusce a odio eget ante consequat volutpat. Nullam at lorem ut dui ornare aliquam nec non justo. Mauris turpis eros, blandit eget elementum ullamcorper, molestie vel quam. Aenean pretium interdum ligula ac fringilla. Aenean et felis lorem. Ut id lectus quis risus finibus condimentum.
Curabitur aliquet quis justo sed posuere. Donec eu libero eget mi ullamcorper placerat. Etiam massa mi, lobortis eget fermentum in, facilisis vel lectus. In hac habitasse platea dictumst. Nam laoreet sodales metus, nec finibus nulla volutpat bibendum. Aenean ut vulputate dolor, vitae venenatis est. Fusce ipsum libero, laoreet sit amet justo at, auctor tristique arcu. Praesent pellentesque efficitur velit non accumsan. Proin fermentum tempus ante, at eleifend lectus fringilla sit amet. Nunc et diam ac velit tristique malesuada a id mauris. Sed euismod turpis lacus, a maximus dolor semper in. Nulla mauris tortor, dignissim vel mauris sed, efficitur ultrices nulla. Donec sed molestie libero. Quisque nec congue eros, ut consequat lectus. Suspendisse vitae tortor sapien. Sed vitae pellentesque dolor.
Suspendisse dictum velit metus, vel mollis erat imperdiet ut. Mauris et sapien eleifend, malesuada ante vehicula, ornare tellus. Integer condimentum mattis risus, nec luctus lacus convallis a. Phasellus bibendum consequat nisi, ut consequat dui bibendum a. Sed venenatis lobortis turpis, a venenatis ex sodales eu. Duis sit amet sem dui. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla vitae tellus a ex tristique mattis. Quisque viverra vitae turpis accumsan imperdiet. Cras nunc erat, commodo et malesuada at, vulputate in lorem. Fusce tempor venenatis dui consequat auctor. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Suspendisse potenti.
Fusce et ligula sem. In condimentum dolor metus, eu luctus justo ultrices nec. Aenean vitae ligula non erat tincidunt blandit. Pellentesque luctus tellus ante, mollis cursus arcu posuere a. Ut sed lacus suscipit, dapibus quam non, elementum purus. Nullam sed venenatis tortor, id luctus lacus. Suspendisse orci felis, porta et sagittis at, posuere vitae lectus.
Cras in ligula ut mi viverra efficitur at sed erat. Donec congue suscipit orci, eu volutpat augue ultricies non. Maecenas imperdiet tincidunt commodo. Cras quis vulputate urna, et malesuada lorem. Donec id convallis nibh, non congue lacus. Curabitur et scelerisque nisl. Maecenas vestibulum elit ipsum, in posuere tortor placerat sit amet. Nullam nec ex eget libero mattis commodo a at leo. Vestibulum aliquet, eros quis facilisis aliquam, mi arcu aliquet nisl, in tempus massa sapien at tortor.
Maecenas et dolor sed sapien lacinia fringilla eget et nibh. Aenean viverra urna sit amet lobortis vestibulum. Aliquam vehicula rutrum magna ut aliquam. Maecenas pharetra volutpat porttitor. In id ultricies sapien, a accumsan lectus. Fusce in elit a ex auctor rutrum sit amet ac lorem. Mauris eu mi a nisl vehicula mattis. Donec dictum velit nec libero bibendum, in volutpat metus viverra. Vivamus eget sollicitudin nunc, ac vestibulum erat. Sed dolor risus, semper nec lorem vitae, vehicula molestie purus. Quisque ac lectus iaculis, ultrices leo sit amet, mattis erat. Curabitur lorem mauris, vestibulum vel risus eu, molestie facilisis elit. Pellentesque habitant morbi tristique senectus et netus et volutpat.

View File

@@ -0,0 +1,71 @@
request = require "request"
Settings = require "settings-sharelatex"
async = require("async")
fs = require("fs")
_ = require("underscore")
concurentCompiles = 5
totalCompiles = 50
buildUrl = (path) -> "http://#{Settings.internal.clsi.host}:#{Settings.internal.clsi.port}/#{path}"
mainTexContent = fs.readFileSync("./bulk.tex", "utf-8")
compileTimes = []
failedCount = 0
getAverageCompileTime = ->
totalTime = _.reduce compileTimes, (sum, time)->
sum + time
, 0
return totalTime / compileTimes.length
makeRequest = (compileNumber, callback)->
bulkBodyCount = 7
bodyContent = ""
while --bulkBodyCount
bodyContent = bodyContent+=mainTexContent
startTime = new Date()
request.post {
url: buildUrl("project/loadcompile-#{compileNumber}/compile")
json:
compile:
resources: [
path: "main.tex"
content: """
\\documentclass{article}
\\begin{document}
#{bodyContent}
\\end{document}
"""
]
}, (err, response, body)->
if response.statusCode != 200
failedCount++
return callback("compile #{compileNumber} failed")
if err?
failedCount++
return callback("failed")
totalTime = new Date() - startTime
console.log totalTime+"ms"
compileTimes.push(totalTime)
callback(err)
jobs = _.map [1..totalCompiles], (i)->
return (cb)->
makeRequest(i, cb)
startTime = new Date()
async.parallelLimit jobs, concurentCompiles, (err)->
if err?
console.error err
console.log("total time taken = #{(new Date() - startTime)/1000}s")
console.log("total compiles = #{totalCompiles}")
console.log("concurent compiles = #{concurentCompiles}")
console.log("average time = #{getAverageCompileTime()/1000}s")
console.log("max time = #{_.max(compileTimes)/1000}s")
console.log("min time = #{_.min(compileTimes)/1000}s")
console.log("total failures = #{failedCount}")

View File

@@ -1,24 +1,53 @@
chai = require("chai")
chai.should()
chai.should() unless Object.prototype.should?
expect = chai.expect
request = require "request"
Settings = require "settings-sharelatex"
buildUrl = (path) -> "http://#{Settings.internal.clsi.host}:#{Settings.internal.clsi.port}/#{path}"
url = buildUrl("project/smoketest-#{process.pid}/compile")
describe "Running a compile", ->
before (done) ->
request.post {
url: buildUrl("project/smoketest/compile")
url: url
json:
compile:
resources: [
path: "main.tex"
content: """
\\documentclass{article}
\\begin{document}
Hello world
\\end{document}
% Membrane-like surface
% Author: Yotam Avital
\\documentclass{article}
\\usepackage{tikz}
\\usetikzlibrary{calc,fadings,decorations.pathreplacing}
\\begin{document}
\\begin{tikzpicture}
\\def\\nuPi{3.1459265}
\\foreach \\i in {5,4,...,2}{% This one doesn't matter
\\foreach \\j in {3,2,...,0}{% This will crate a membrane
% with the front lipids visible
% top layer
\\pgfmathsetmacro{\\dx}{rand*0.1}% A random variance in the x coordinate
\\pgfmathsetmacro{\\dy}{rand*0.1}% A random variance in the y coordinate,
% gives a hight fill to the lipid
\\pgfmathsetmacro{\\rot}{rand*0.1}% A random variance in the
% molecule orientation
\\shade[ball color=red] ({\\i+\\dx+\\rot},{0.5*\\j+\\dy+0.4*sin(\\i*\\nuPi*10)}) circle(0.45);
\\shade[ball color=gray] (\\i+\\dx,{0.5*\\j+\\dy+0.4*sin(\\i*\\nuPi*10)-0.9}) circle(0.45);
\\shade[ball color=gray] (\\i+\\dx-\\rot,{0.5*\\j+\\dy+0.4*sin(\\i*\\nuPi*10)-1.8}) circle(0.45);
% bottom layer
\\pgfmathsetmacro{\\dx}{rand*0.1}
\\pgfmathsetmacro{\\dy}{rand*0.1}
\\pgfmathsetmacro{\\rot}{rand*0.1}
\\shade[ball color=gray] (\\i+\\dx+\\rot,{0.5*\\j+\\dy+0.4*sin(\\i*\\nuPi*10)-2.8}) circle(0.45);
\\shade[ball color=gray] (\\i+\\dx,{0.5*\\j+\\dy+0.4*sin(\\i*\\nuPi*10)-3.7}) circle(0.45);
\\shade[ball color=red] (\\i+\\dx-\\rot,{0.5*\\j+\\dy+0.4*sin(\\i*\\nuPi*10)-4.6}) circle(0.45);
}
}
\\end{tikzpicture}
\\end{document}
"""
]
}, (@error, @response, @body) =>

View File

@@ -36,17 +36,20 @@ describe "CompileController", ->
@output_files = [{
path: "output.pdf"
type: "pdf"
build: 1234
}, {
path: "output.log"
type: "log"
build: 1234
}]
@RequestParser.parse = sinon.stub().callsArgWith(1, null, @request)
@ProjectPersistenceManager.markProjectAsJustAccessed = sinon.stub().callsArg(1)
@res.status = sinon.stub().returnsThis()
@res.send = sinon.stub()
describe "successfully", ->
beforeEach ->
@CompileManager.doCompile = sinon.stub().callsArgWith(1, null, @output_files)
@CompileManager.doCompileWithLock = sinon.stub().callsArgWith(1, null, @output_files)
@CompileController.compile @req, @res
it "should parse the request", ->
@@ -55,7 +58,7 @@ describe "CompileController", ->
.should.equal true
it "should run the compile for the specified project", ->
@CompileManager.doCompile
@CompileManager.doCompileWithLock
.calledWith(@request_with_project_id)
.should.equal true
@@ -65,25 +68,29 @@ describe "CompileController", ->
.should.equal true
it "should return the JSON response", ->
@res.status.calledWith(200).should.equal true
@res.send
.calledWith(200,
.calledWith(
compile:
status: "success"
error: null
outputFiles: @output_files.map (file) =>
url: "#{@Settings.apis.clsi.url}/project/#{@project_id}/output/#{file.path}"
url: "#{@Settings.apis.clsi.url}/project/#{@project_id}/build/#{file.build}/output/#{file.path}"
path: file.path
type: file.type
build: file.build
)
.should.equal true
describe "with an error", ->
beforeEach ->
@CompileManager.doCompile = sinon.stub().callsArgWith(1, new Error(@message = "error message"), null)
@CompileManager.doCompileWithLock = sinon.stub().callsArgWith(1, new Error(@message = "error message"), null)
@CompileController.compile @req, @res
it "should return the JSON response with the error", ->
@res.status.calledWith(500).should.equal true
@res.send
.calledWith(500,
.calledWith(
compile:
status: "error"
error: @message
@@ -95,12 +102,13 @@ describe "CompileController", ->
beforeEach ->
@error = new Error(@message = "container timed out")
@error.timedout = true
@CompileManager.doCompile = sinon.stub().callsArgWith(1, @error, null)
@CompileManager.doCompileWithLock = sinon.stub().callsArgWith(1, @error, null)
@CompileController.compile @req, @res
it "should return the JSON response with the timeout status", ->
@res.status.calledWith(200).should.equal true
@res.send
.calledWith(200,
.calledWith(
compile:
status: "timedout"
error: @message
@@ -110,12 +118,13 @@ describe "CompileController", ->
describe "when the request returns no output files", ->
beforeEach ->
@CompileManager.doCompile = sinon.stub().callsArgWith(1, null, [])
@CompileManager.doCompileWithLock = sinon.stub().callsArgWith(1, null, [])
@CompileController.compile @req, @res
it "should return the JSON response with the failure status", ->
@res.status.calledWith(200).should.equal true
@res.send
.calledWith(200,
.calledWith(
compile:
error: null
status: "failure"
@@ -137,12 +146,12 @@ describe "CompileController", ->
column: @column.toString()
@res.send = sinon.stub()
@CompileManager.syncFromCode = sinon.stub().callsArgWith(4, null, @pdfPositions = ["mock-positions"])
@CompileManager.syncFromCode = sinon.stub().callsArgWith(5, null, @pdfPositions = ["mock-positions"])
@CompileController.syncFromCode @req, @res, @next
it "should find the corresponding location in the PDF", ->
@CompileManager.syncFromCode
.calledWith(@project_id, @file, @line, @column)
.calledWith(@project_id, undefined, @file, @line, @column)
.should.equal true
it "should return the positions", ->
@@ -166,12 +175,12 @@ describe "CompileController", ->
v: @v.toString()
@res.send = sinon.stub()
@CompileManager.syncFromPdf = sinon.stub().callsArgWith(4, null, @codePositions = ["mock-positions"])
@CompileManager.syncFromPdf = sinon.stub().callsArgWith(5, null, @codePositions = ["mock-positions"])
@CompileController.syncFromPdf @req, @res, @next
it "should find the corresponding location in the code", ->
@CompileManager.syncFromPdf
.calledWith(@project_id, @page, @h, @v)
.calledWith(@project_id, undefined, @page, @h, @v)
.should.equal true
it "should return the positions", ->
@@ -181,3 +190,28 @@ describe "CompileController", ->
)
.should.equal true
describe "wordcount", ->
beforeEach ->
@file = "main.tex"
@project_id = "mock-project-id"
@req.params =
project_id: @project_id
@req.query =
file: @file
image: @image = "example.com/image"
@res.send = sinon.stub()
@CompileManager.wordcount = sinon.stub().callsArgWith(4, null, @texcount = ["mock-texcount"])
@CompileController.wordcount @req, @res, @next
it "should return the word count of a file", ->
@CompileManager.wordcount
.calledWith(@project_id, undefined, @file, @image)
.should.equal true
it "should return the texcount info", ->
@res.send
.calledWith(JSON.stringify
texcount: @texcount
)
.should.equal true

View File

@@ -12,11 +12,68 @@ describe "CompileManager", ->
"./LatexRunner": @LatexRunner = {}
"./ResourceWriter": @ResourceWriter = {}
"./OutputFileFinder": @OutputFileFinder = {}
"./OutputCacheManager": @OutputCacheManager = {}
"settings-sharelatex": @Settings = { path: compilesDir: "/compiles/dir" }
"logger-sharelatex": @logger = { log: sinon.stub() }
"logger-sharelatex": @logger = { log: sinon.stub() , info:->}
"child_process": @child_process = {}
"./CommandRunner": @CommandRunner = {}
"./DraftModeManager": @DraftModeManager = {}
"./TikzManager": @TikzManager = {}
"./LockManager": @LockManager = {}
"fs": @fs = {}
"fs-extra": @fse = { ensureDir: sinon.stub().callsArg(1) }
@callback = sinon.stub()
describe "doCompileWithLock", ->
beforeEach ->
@request =
resources: @resources = "mock-resources"
project_id: @project_id = "project-id-123"
user_id: @user_id = "1234"
@output_files = ["foo", "bar"]
@Settings.compileDir = "compiles"
@compileDir = "#{@Settings.path.compilesDir}/#{@project_id}-#{@user_id}"
@CompileManager.doCompile = sinon.stub().callsArgWith(1, null, @output_files)
@LockManager.runWithLock = (lockFile, runner, callback) ->
runner (err, result...) ->
callback(err, result...)
describe "when the project is not locked", ->
beforeEach ->
@CompileManager.doCompileWithLock @request, @callback
it "should ensure that the compile directory exists", ->
@fse.ensureDir.calledWith(@compileDir)
.should.equal true
it "should call doCompile with the request", ->
@CompileManager.doCompile
.calledWith(@request)
.should.equal true
it "should call the callback with the output files", ->
@callback.calledWithExactly(null, @output_files)
.should.equal true
describe "when the project is locked", ->
beforeEach ->
@error = new Error("locked")
@LockManager.runWithLock = (lockFile, runner, callback) =>
callback(@error)
@CompileManager.doCompileWithLock @request, @callback
it "should ensure that the compile directory exists", ->
@fse.ensureDir.calledWith(@compileDir)
.should.equal true
it "should not call doCompile with the request", ->
@CompileManager.doCompile
.called.should.equal false
it "should call the callback with the error", ->
@callback.calledWithExactly(@error)
.should.equal true
describe "doCompile", ->
beforeEach ->
@output_files = [{
@@ -26,56 +83,125 @@ describe "CompileManager", ->
path: "output.pdf"
type: "pdf"
}]
@build_files = [{
path: "output.log"
type: "log"
build: 1234
}, {
path: "output.pdf"
type: "pdf"
build: 1234
}]
@request =
resources: @resources = "mock-resources"
rootResourcePath: @rootResourcePath = "main.tex"
project_id: @project_id = "project-id-123"
user_id: @user_id = "1234"
compiler: @compiler = "pdflatex"
timeout: @timeout = 42000
imageName: @image = "example.com/image"
@env = {}
@Settings.compileDir = "compiles"
@compileDir = "#{@Settings.path.compilesDir}/#{@project_id}"
@ResourceWriter.syncResourcesToDisk = sinon.stub().callsArg(3)
@compileDir = "#{@Settings.path.compilesDir}/#{@project_id}-#{@user_id}"
@ResourceWriter.syncResourcesToDisk = sinon.stub().callsArgWith(2, null, @resources)
@LatexRunner.runLatex = sinon.stub().callsArg(2)
@OutputFileFinder.findOutputFiles = sinon.stub().callsArgWith(2, null, @output_files)
@CompileManager.doCompile @request, @callback
@OutputCacheManager.saveOutputFiles = sinon.stub().callsArgWith(2, null, @build_files)
@DraftModeManager.injectDraftMode = sinon.stub().callsArg(1)
@TikzManager.checkMainFile = sinon.stub().callsArg(3, false)
describe "normally", ->
beforeEach ->
@CompileManager.doCompile @request, @callback
it "should write the resources to disk", ->
@ResourceWriter.syncResourcesToDisk
.calledWith(@project_id, @resources, @compileDir)
.should.equal true
it "should write the resources to disk", ->
@ResourceWriter.syncResourcesToDisk
.calledWith(@request, @compileDir)
.should.equal true
it "should run LaTeX", ->
@LatexRunner.runLatex
.calledWith(@project_id, {
directory: @compileDir
mainFile: @rootResourcePath
compiler: @compiler
timeout: @timeout
})
.should.equal true
it "should run LaTeX", ->
@LatexRunner.runLatex
.calledWith("#{@project_id}-#{@user_id}", {
directory: @compileDir
mainFile: @rootResourcePath
compiler: @compiler
timeout: @timeout
image: @image
environment: @env
})
.should.equal true
it "should find the output files", ->
@OutputFileFinder.findOutputFiles
.calledWith(@resources, @compileDir)
.should.equal true
it "should find the output files", ->
@OutputFileFinder.findOutputFiles
.calledWith(@resources, @compileDir)
.should.equal true
it "should return the output files", ->
@callback.calledWith(null, @output_files).should.equal true
it "should return the output files", ->
@callback.calledWith(null, @build_files).should.equal true
it "should not inject draft mode by default", ->
@DraftModeManager.injectDraftMode.called.should.equal false
describe "with draft mode", ->
beforeEach ->
@request.draft = true
@CompileManager.doCompile @request, @callback
it "should inject the draft mode header", ->
@DraftModeManager.injectDraftMode
.calledWith(@compileDir + "/" + @rootResourcePath)
.should.equal true
describe "with a check option", ->
beforeEach ->
@request.check = "error"
@CompileManager.doCompile @request, @callback
it "should run chktex", ->
@LatexRunner.runLatex
.calledWith("#{@project_id}-#{@user_id}", {
directory: @compileDir
mainFile: @rootResourcePath
compiler: @compiler
timeout: @timeout
image: @image
environment: {'CHKTEX_OPTIONS': '-nall -e9 -e10 -w15 -w16', 'CHKTEX_EXIT_ON_ERROR':1, 'CHKTEX_ULIMIT_OPTIONS': '-t 5 -v 64000'}
})
.should.equal true
describe "with a knitr file and check options", ->
beforeEach ->
@request.rootResourcePath = "main.Rtex"
@request.check = "error"
@CompileManager.doCompile @request, @callback
it "should not run chktex", ->
@LatexRunner.runLatex
.calledWith("#{@project_id}-#{@user_id}", {
directory: @compileDir
mainFile: "main.Rtex"
compiler: @compiler
timeout: @timeout
image: @image
environment: @env
})
.should.equal true
describe "clearProject", ->
describe "succesfully", ->
beforeEach ->
@Settings.compileDir = "compiles"
@fs.lstat = sinon.stub().callsArgWith(1, null,{isDirectory: ()->true})
@proc = new EventEmitter()
@proc.stdout = new EventEmitter()
@proc.stderr = new EventEmitter()
@child_process.spawn = sinon.stub().returns(@proc)
@CompileManager.clearProject @project_id, @callback
@CompileManager.clearProject @project_id, @user_id, @callback
@proc.emit "close", 0
it "should remove the project directory", ->
@child_process.spawn
.calledWith("rm", ["-r", "#{@Settings.path.compilesDir}/#{@project_id}"])
.calledWith("rm", ["-r", "#{@Settings.path.compilesDir}/#{@project_id}-#{@user_id}"])
.should.equal true
it "should call the callback", ->
@@ -84,17 +210,18 @@ describe "CompileManager", ->
describe "with a non-success status code", ->
beforeEach ->
@Settings.compileDir = "compiles"
@fs.lstat = sinon.stub().callsArgWith(1, null,{isDirectory: ()->true})
@proc = new EventEmitter()
@proc.stdout = new EventEmitter()
@proc.stderr = new EventEmitter()
@child_process.spawn = sinon.stub().returns(@proc)
@CompileManager.clearProject @project_id, @callback
@CompileManager.clearProject @project_id, @user_id, @callback
@proc.stderr.emit "data", @error = "oops"
@proc.emit "close", 1
it "should remove the project directory", ->
@child_process.spawn
.calledWith("rm", ["-r", "#{@Settings.path.compilesDir}/#{@project_id}"])
.calledWith("rm", ["-r", "#{@Settings.path.compilesDir}/#{@project_id}-#{@user_id}"])
.should.equal true
it "should call the callback with an error from the stderr", ->
@@ -102,7 +229,7 @@ describe "CompileManager", ->
.calledWith(new Error())
.should.equal true
@callback.args[0][0].message.should.equal "rm -r #{@Settings.path.compilesDir}/#{@project_id} failed: #{@error}"
@callback.args[0][0].message.should.equal "rm -r #{@Settings.path.compilesDir}/#{@project_id}-#{@user_id} failed: #{@error}"
describe "syncing", ->
beforeEach ->
@@ -115,17 +242,18 @@ describe "CompileManager", ->
@column = 3
@file_name = "main.tex"
@child_process.execFile = sinon.stub()
@Settings.path.synctexBaseDir = (project_id) => "#{@Settings.path.compilesDir}/#{@project_id}"
@Settings.path.synctexBaseDir = (project_id) => "#{@Settings.path.compilesDir}/#{@project_id}-#{@user_id}"
describe "syncFromCode", ->
beforeEach ->
@fs.stat = sinon.stub().callsArgWith(1, null,{isFile: ()->true})
@child_process.execFile.callsArgWith(3, null, @stdout = "NODE\t#{@page}\t#{@h}\t#{@v}\t#{@width}\t#{@height}\n", "")
@CompileManager.syncFromCode @project_id, @file_name, @line, @column, @callback
@CompileManager.syncFromCode @project_id, @user_id, @file_name, @line, @column, @callback
it "should execute the synctex binary", ->
bin_path = Path.resolve(__dirname + "/../../../bin/synctex")
synctex_path = "#{@Settings.path.compilesDir}/#{@project_id}/output.pdf"
file_path = "#{@Settings.path.compilesDir}/#{@project_id}/#{@file_name}"
synctex_path = "#{@Settings.path.compilesDir}/#{@project_id}-#{@user_id}/output.pdf"
file_path = "#{@Settings.path.compilesDir}/#{@project_id}-#{@user_id}/#{@file_name}"
@child_process.execFile
.calledWith(bin_path, ["code", synctex_path, file_path, @line, @column], timeout: 10000)
.should.equal true
@@ -143,12 +271,13 @@ describe "CompileManager", ->
describe "syncFromPdf", ->
beforeEach ->
@child_process.execFile.callsArgWith(3, null, @stdout = "NODE\t#{@Settings.path.compilesDir}/#{@project_id}/#{@file_name}\t#{@line}\t#{@column}\n", "")
@CompileManager.syncFromPdf @project_id, @page, @h, @v, @callback
@fs.stat = sinon.stub().callsArgWith(1, null,{isFile: ()->true})
@child_process.execFile.callsArgWith(3, null, @stdout = "NODE\t#{@Settings.path.compilesDir}/#{@project_id}-#{@user_id}/#{@file_name}\t#{@line}\t#{@column}\n", "")
@CompileManager.syncFromPdf @project_id, @user_id, @page, @h, @v, @callback
it "should execute the synctex binary", ->
bin_path = Path.resolve(__dirname + "/../../../bin/synctex")
synctex_path = "#{@Settings.path.compilesDir}/#{@project_id}/output.pdf"
synctex_path = "#{@Settings.path.compilesDir}/#{@project_id}-#{@user_id}/output.pdf"
@child_process.execFile
.calledWith(bin_path, ["pdf", synctex_path, @page, @h, @v], timeout: 10000)
.should.equal true
@@ -160,4 +289,43 @@ describe "CompileManager", ->
line: @line
column: @column
}])
.should.equal true
.should.equal true
describe "wordcount", ->
beforeEach ->
@CommandRunner.run = sinon.stub().callsArg(6)
@fs.readFile = sinon.stub().callsArgWith(2, null, @stdout = "Encoding: ascii\nWords in text: 2")
@callback = sinon.stub()
@project_id = "project-id-123"
@timeout = 10 * 1000
@file_name = "main.tex"
@Settings.path.compilesDir = "/local/compile/directory"
@image = "example.com/image"
@CompileManager.wordcount @project_id, @user_id, @file_name, @image, @callback
it "should run the texcount command", ->
@directory = "#{@Settings.path.compilesDir}/#{@project_id}-#{@user_id}"
@file_path = "$COMPILE_DIR/#{@file_name}"
@command =[ "texcount", "-nocol", "-inc", @file_path, "-out=" + @file_path + ".wc"]
@CommandRunner.run
.calledWith("#{@project_id}-#{@user_id}", @command, @directory, @image, @timeout, {})
.should.equal true
it "should call the callback with the parsed output", ->
@callback
.calledWith(null, {
encode: "ascii"
textWords: 2
headWords: 0
outside: 0
headers: 0
elements: 0
mathInline: 0
mathDisplay: 0
errors: 0
messages: ""
})
.should.equal true

View File

@@ -0,0 +1,55 @@
SandboxedModule = require('sandboxed-module')
sinon = require('sinon')
require('chai').should()
modulePath = require('path').join __dirname, '../../../app/js/ContentTypeMapper'
describe 'ContentTypeMapper', ->
beforeEach ->
@ContentTypeMapper = SandboxedModule.require modulePath
describe 'map', ->
it 'should map .txt to text/plain', ->
content_type = @ContentTypeMapper.map('example.txt')
content_type.should.equal 'text/plain'
it 'should map .csv to text/csv', ->
content_type = @ContentTypeMapper.map('example.csv')
content_type.should.equal 'text/csv'
it 'should map .pdf to application/pdf', ->
content_type = @ContentTypeMapper.map('example.pdf')
content_type.should.equal 'application/pdf'
it 'should fall back to octet-stream', ->
content_type = @ContentTypeMapper.map('example.unknown')
content_type.should.equal 'application/octet-stream'
describe 'coercing web files to plain text', ->
it 'should map .js to plain text', ->
content_type = @ContentTypeMapper.map('example.js')
content_type.should.equal 'text/plain'
it 'should map .html to plain text', ->
content_type = @ContentTypeMapper.map('example.html')
content_type.should.equal 'text/plain'
it 'should map .css to plain text', ->
content_type = @ContentTypeMapper.map('example.css')
content_type.should.equal 'text/plain'
describe 'image files', ->
it 'should map .png to image/png', ->
content_type = @ContentTypeMapper.map('example.png')
content_type.should.equal 'image/png'
it 'should map .jpeg to image/jpeg', ->
content_type = @ContentTypeMapper.map('example.jpeg')
content_type.should.equal 'image/jpeg'
it 'should map .svg to text/plain to protect against XSS (SVG can execute JS)', ->
content_type = @ContentTypeMapper.map('example.svg')
content_type.should.equal 'text/plain'

View File

@@ -0,0 +1,61 @@
SandboxedModule = require('sandboxed-module')
sinon = require('sinon')
require('chai').should()
modulePath = require('path').join __dirname, '../../../app/js/DraftModeManager'
describe 'DraftModeManager', ->
beforeEach ->
@DraftModeManager = SandboxedModule.require modulePath, requires:
"fs": @fs = {}
"logger-sharelatex": @logger = {log: () ->}
describe "_injectDraftOption", ->
it "should add draft option into documentclass with existing options", ->
@DraftModeManager
._injectDraftOption('''
\\documentclass[a4paper,foo=bar]{article}
''')
.should.equal('''
\\documentclass[draft,a4paper,foo=bar]{article}
''')
it "should add draft option into documentclass with no options", ->
@DraftModeManager
._injectDraftOption('''
\\documentclass{article}
''')
.should.equal('''
\\documentclass[draft]{article}
''')
describe "injectDraftMode", ->
beforeEach ->
@filename = "/mock/filename.tex"
@callback = sinon.stub()
content = '''
\\documentclass{article}
\\begin{document}
Hello world
\\end{document}
'''
@fs.readFile = sinon.stub().callsArgWith(2, null, content)
@fs.writeFile = sinon.stub().callsArg(2)
@DraftModeManager.injectDraftMode @filename, @callback
it "should read the file", ->
@fs.readFile
.calledWith(@filename, "utf8")
.should.equal true
it "should write the modified file", ->
@fs.writeFile
.calledWith(@filename, """
\\documentclass[draft]{article}
\\begin{document}
Hello world
\\end{document}
""")
.should.equal true
it "should call the callback", ->
@callback.called.should.equal true

View File

@@ -19,12 +19,14 @@ describe "LatexRunner", ->
@directory = "/local/compile/directory"
@mainFile = "main-file.tex"
@compiler = "pdflatex"
@image = "example.com/image"
@callback = sinon.stub()
@project_id = "project-id-123"
@env = {'foo': '123'}
describe "runLatex", ->
beforeEach ->
@CommandRunner.run = sinon.stub().callsArg(4)
@CommandRunner.run = sinon.stub().callsArg(6)
describe "normally", ->
beforeEach ->
@@ -33,11 +35,13 @@ describe "LatexRunner", ->
mainFile: @mainFile
compiler: @compiler
timeout: @timeout = 42000
image: @image
environment: @env
@callback
it "should run the latex command", ->
@CommandRunner.run
.calledWith(@project_id, sinon.match.any, @directory, @timeout)
.calledWith(@project_id, sinon.match.any, @directory, @image, @timeout, @env)
.should.equal true
describe "with an .Rtex main file", ->
@@ -46,6 +50,7 @@ describe "LatexRunner", ->
directory: @directory
mainFile: "main-file.Rtex"
compiler: @compiler
image: @image
timeout: @timeout = 42000
@callback

View File

@@ -0,0 +1,54 @@
SandboxedModule = require('sandboxed-module')
sinon = require('sinon')
require('chai').should()
modulePath = require('path').join __dirname, '../../../app/js/LockManager'
Path = require "path"
Errors = require "../../../app/js/Errors"
describe "LockManager", ->
beforeEach ->
@LockManager = SandboxedModule.require modulePath, requires:
"settings-sharelatex": {}
"logger-sharelatex": @logger = { log: sinon.stub(), error: sinon.stub() }
"lockfile": @Lockfile = {}
@lockFile = "/local/compile/directory/.project-lock"
describe "runWithLock", ->
beforeEach ->
@runner = sinon.stub().callsArgWith(0, null, "foo", "bar")
@callback = sinon.stub()
describe "normally", ->
beforeEach ->
@Lockfile.lock = sinon.stub().callsArgWith(2, null)
@Lockfile.unlock = sinon.stub().callsArgWith(1, null)
@LockManager.runWithLock @lockFile, @runner, @callback
it "should run the compile", ->
@runner
.calledWith()
.should.equal true
it "should call the callback with the response from the compile", ->
@callback
.calledWithExactly(null, "foo", "bar")
.should.equal true
describe "when the project is locked", ->
beforeEach ->
@error = new Error()
@error.code = "EEXIST"
@Lockfile.lock = sinon.stub().callsArgWith(2,@error)
@Lockfile.unlock = sinon.stub().callsArgWith(1, null)
@LockManager.runWithLock @lockFile, @runner, @callback
it "should not run the compile", ->
@runner
.called
.should.equal false
it "should return an error", ->
error = new Errors.AlreadyCompilingError()
@callback
.calledWithExactly(error)
.should.equal true

View File

@@ -4,29 +4,26 @@ require('chai').should()
modulePath = require('path').join __dirname, '../../../app/js/OutputFileFinder'
path = require "path"
expect = require("chai").expect
EventEmitter = require("events").EventEmitter
describe "OutputFileFinder", ->
beforeEach ->
@OutputFileFinder = SandboxedModule.require modulePath, requires:
"fs": @fs = {}
"wrench": @wrench = {}
"child_process": spawn: @spawn = sinon.stub()
"logger-sharelatex": { log: sinon.stub(), warn: sinon.stub() }
@directory = "/test/dir"
@callback = sinon.stub()
describe "findOutputFiles", ->
beforeEach ->
@resource_path = "resource/path.tex"
@output_paths = ["output.pdf", "extra", "extra/file.tex"]
@output_paths = ["output.pdf", "extra/file.tex"]
@all_paths = @output_paths.concat [@resource_path]
@resources = [
path: @resource_path = "resource/path.tex"
]
@OutputFileFinder._isDirectory = (dirPath, callback = (error, directory) ->) =>
callback null, dirPath == path.join(@directory, "extra")
@wrench.readdirRecursive = (dir, callback) =>
callback(null, [@resource_path].concat(@output_paths))
callback(null, null)
sinon.spy @wrench, "readdirRecursive"
@OutputFileFinder._getAllFiles = sinon.stub().callsArgWith(1, null, @all_paths)
@OutputFileFinder.findOutputFiles @resources, @directory, (error, @outputFiles) =>
it "should only return the output files, not directories or resource paths", ->
@@ -37,5 +34,35 @@ describe "OutputFileFinder", ->
path: "extra/file.tex",
type: "tex"
}]
describe "_getAllFiles", ->
beforeEach ->
@proc = new EventEmitter()
@proc.stdout = new EventEmitter()
@spawn.returns @proc
@directory = "/base/dir"
@OutputFileFinder._getAllFiles @directory, @callback
describe "successfully", ->
beforeEach ->
@proc.stdout.emit(
"data",
["/base/dir/main.tex", "/base/dir/chapters/chapter1.tex"].join("\n") + "\n"
)
@proc.emit "close", 0
it "should call the callback with the relative file paths", ->
@callback.calledWith(
null,
["main.tex", "chapters/chapter1.tex"]
).should.equal true
describe "when the directory doesn't exist", ->
beforeEach ->
@proc.emit "close", 1
it "should call the callback with a blank array", ->
@callback.calledWith(
null,
[]
).should.equal true

View File

@@ -0,0 +1,103 @@
SandboxedModule = require('sandboxed-module')
sinon = require('sinon')
require('chai').should()
modulePath = require('path').join __dirname, '../../../app/js/OutputFileOptimiser'
path = require "path"
expect = require("chai").expect
EventEmitter = require("events").EventEmitter
describe "OutputFileOptimiser", ->
beforeEach ->
@OutputFileOptimiser = SandboxedModule.require modulePath, requires:
"fs": @fs = {}
"path": @Path = {}
"child_process": spawn: @spawn = sinon.stub()
"logger-sharelatex": { log: sinon.stub(), warn: sinon.stub() }
"./Metrics" : {}
@directory = "/test/dir"
@callback = sinon.stub()
describe "optimiseFile", ->
beforeEach ->
@src = "./output.pdf"
@dst = "./output.pdf"
describe "when the file is not a pdf file", ->
beforeEach (done)->
@src = "./output.log"
@OutputFileOptimiser.checkIfPDFIsOptimised = sinon.stub().callsArgWith(1, null, false)
@OutputFileOptimiser.optimisePDF = sinon.stub().callsArgWith(2, null)
@OutputFileOptimiser.optimiseFile @src, @dst, done
it "should not check if the file is optimised", ->
@OutputFileOptimiser.checkIfPDFIsOptimised.calledWith(@src).should.equal false
it "should not optimise the file", ->
@OutputFileOptimiser.optimisePDF.calledWith(@src, @dst).should.equal false
describe "when the pdf file is not optimised", ->
beforeEach (done) ->
@OutputFileOptimiser.checkIfPDFIsOptimised = sinon.stub().callsArgWith(1, null, false)
@OutputFileOptimiser.optimisePDF = sinon.stub().callsArgWith(2, null)
@OutputFileOptimiser.optimiseFile @src, @dst, done
it "should check if the pdf is optimised", ->
@OutputFileOptimiser.checkIfPDFIsOptimised.calledWith(@src).should.equal true
it "should optimise the pdf", ->
@OutputFileOptimiser.optimisePDF.calledWith(@src, @dst).should.equal true
describe "when the pdf file is optimised", ->
beforeEach (done) ->
@OutputFileOptimiser.checkIfPDFIsOptimised = sinon.stub().callsArgWith(1, null, true)
@OutputFileOptimiser.optimisePDF = sinon.stub().callsArgWith(2, null)
@OutputFileOptimiser.optimiseFile @src, @dst, done
it "should check if the pdf is optimised", ->
@OutputFileOptimiser.checkIfPDFIsOptimised.calledWith(@src).should.equal true
it "should not optimise the pdf", ->
@OutputFileOptimiser.optimisePDF.calledWith(@src, @dst).should.equal false
describe "checkIfPDFISOptimised", ->
beforeEach () ->
@callback = sinon.stub()
@fd = 1234
@fs.open = sinon.stub().yields(null, @fd)
@fs.read = sinon.stub().withArgs(@fd).yields(null, 100, new Buffer("hello /Linearized 1"))
@fs.close = sinon.stub().withArgs(@fd).yields(null)
@OutputFileOptimiser.checkIfPDFIsOptimised @src, @callback
describe "for a linearised file", ->
beforeEach () ->
@fs.read = sinon.stub().withArgs(@fd).yields(null, 100, new Buffer("hello /Linearized 1"))
@OutputFileOptimiser.checkIfPDFIsOptimised @src, @callback
it "should open the file", ->
@fs.open.calledWith(@src, "r").should.equal true
it "should read the header", ->
@fs.read.calledWith(@fd).should.equal true
it "should close the file", ->
@fs.close.calledWith(@fd).should.equal true
it "should call the callback with a true result", ->
@callback.calledWith(null, true).should.equal true
describe "for an unlinearised file", ->
beforeEach () ->
@fs.read = sinon.stub().withArgs(@fd).yields(null, 100, new Buffer("hello not linearized 1"))
@OutputFileOptimiser.checkIfPDFIsOptimised @src, @callback
it "should open the file", ->
@fs.open.calledWith(@src, "r").should.equal true
it "should read the header", ->
@fs.read.calledWith(@fd).should.equal true
it "should close the file", ->
@fs.close.calledWith(@fd).should.equal true
it "should call the callback with a false result", ->
@callback.calledWith(null, false).should.equal true

View File

@@ -13,6 +13,7 @@ describe "ProjectPersistenceManager", ->
"./db": @db = {}
@callback = sinon.stub()
@project_id = "project-id-123"
@user_id = "1234"
describe "clearExpiredProjects", ->
beforeEach ->
@@ -21,12 +22,13 @@ describe "ProjectPersistenceManager", ->
"project-id-2"
]
@ProjectPersistenceManager._findExpiredProjectIds = sinon.stub().callsArgWith(0, null, @project_ids)
@ProjectPersistenceManager.clearProject = sinon.stub().callsArg(1)
@ProjectPersistenceManager.clearProjectFromCache = sinon.stub().callsArg(1)
@CompileManager.clearExpiredProjects = sinon.stub().callsArg(1)
@ProjectPersistenceManager.clearExpiredProjects @callback
it "should clear each expired project", ->
for project_id in @project_ids
@ProjectPersistenceManager.clearProject
@ProjectPersistenceManager.clearProjectFromCache
.calledWith(project_id)
.should.equal true
@@ -37,8 +39,8 @@ describe "ProjectPersistenceManager", ->
beforeEach ->
@ProjectPersistenceManager._clearProjectFromDatabase = sinon.stub().callsArg(1)
@UrlCache.clearProject = sinon.stub().callsArg(1)
@CompileManager.clearProject = sinon.stub().callsArg(1)
@ProjectPersistenceManager.clearProject @project_id, @callback
@CompileManager.clearProject = sinon.stub().callsArg(2)
@ProjectPersistenceManager.clearProject @project_id, @user_id, @callback
it "should clear the project from the database", ->
@ProjectPersistenceManager._clearProjectFromDatabase
@@ -52,7 +54,7 @@ describe "ProjectPersistenceManager", ->
it "should clear the project compile folder", ->
@CompileManager.clearProject
.calledWith(@project_id)
.calledWith(@project_id, @user_id)
.should.equal true
it "should call the callback", ->

View File

@@ -204,6 +204,51 @@ describe "RequestParser", ->
@callback.calledWith("rootResourcePath attribute should be a string")
.should.equal true
describe "with a root resource path that needs escaping", ->
beforeEach ->
@badPath = "`rm -rf foo`.tex"
@goodPath = "rm -rf foo.tex"
@validRequest.compile.rootResourcePath = @badPath
@validRequest.compile.resources.push {
path: @badPath
date: "12:00 01/02/03"
content: "Hello world"
}
@RequestParser.parse @validRequest, @callback
@data = @callback.args[0][1]
it "should return the escaped resource", ->
@data.rootResourcePath.should.equal @goodPath
it "should also escape the resource path", ->
@data.resources[0].path.should.equal @goodPath
describe "with a root resource path that has a relative path", ->
beforeEach ->
@validRequest.compile.rootResourcePath = "foo/../../bar.tex"
@RequestParser.parse @validRequest, @callback
@data = @callback.args[0][1]
it "should return an error", ->
@callback.calledWith("relative path in root resource")
.should.equal true
describe "with a root resource path that has unescaped + relative path", ->
beforeEach ->
@validRequest.compile.rootResourcePath = "foo/#../bar.tex"
@RequestParser.parse @validRequest, @callback
@data = @callback.args[0][1]
it "should return an error", ->
@callback.calledWith("relative path in root resource")
.should.equal true
describe "with an unknown syncType", ->
beforeEach ->
@validRequest.compile.options.syncType = "unexpected"
@RequestParser.parse @validRequest, @callback
@data = @callback.args[0][1]
it "should return an error", ->
@callback.calledWith("syncType attribute should be one of: full, incremental")
.should.equal true

View File

@@ -0,0 +1,109 @@
SandboxedModule = require('sandboxed-module')
sinon = require('sinon')
should = require('chai').should()
modulePath = require('path').join __dirname, '../../../app/js/ResourceStateManager'
Path = require "path"
Errors = require "../../../app/js/Errors"
describe "ResourceStateManager", ->
beforeEach ->
@ResourceStateManager = SandboxedModule.require modulePath, requires:
"fs": @fs = {}
"logger-sharelatex": {log: sinon.stub(), err: sinon.stub()}
"./SafeReader": @SafeReader = {}
@basePath = "/path/to/write/files/to"
@resources = [
{path: "resource-1-mock"}
{path: "resource-2-mock"}
{path: "resource-3-mock"}
]
@state = "1234567890"
@resourceFileName = "#{@basePath}/.project-sync-state"
@resourceFileContents = "#{@resources[0].path}\n#{@resources[1].path}\n#{@resources[2].path}\nstateHash:#{@state}"
@callback = sinon.stub()
describe "saveProjectState", ->
beforeEach ->
@fs.writeFile = sinon.stub().callsArg(2)
describe "when the state is specified", ->
beforeEach ->
@ResourceStateManager.saveProjectState(@state, @resources, @basePath, @callback)
it "should write the resource list to disk", ->
@fs.writeFile
.calledWith(@resourceFileName, @resourceFileContents)
.should.equal true
it "should call the callback", ->
@callback.called.should.equal true
describe "when the state is undefined", ->
beforeEach ->
@state = undefined
@fs.unlink = sinon.stub().callsArg(1)
@ResourceStateManager.saveProjectState(@state, @resources, @basePath, @callback)
it "should unlink the resource file", ->
@fs.unlink
.calledWith(@resourceFileName)
.should.equal true
it "should not write the resource list to disk", ->
@fs.writeFile.called.should.equal false
it "should call the callback", ->
@callback.called.should.equal true
describe "checkProjectStateMatches", ->
describe "when the state matches", ->
beforeEach ->
@SafeReader.readFile = sinon.stub().callsArgWith(3, null, @resourceFileContents)
@ResourceStateManager.checkProjectStateMatches(@state, @basePath, @callback)
it "should read the resource file", ->
@SafeReader.readFile
.calledWith(@resourceFileName)
.should.equal true
it "should call the callback with the results", ->
@callback.calledWithMatch(null, @resources).should.equal true
describe "when the state does not match", ->
beforeEach ->
@SafeReader.readFile = sinon.stub().callsArgWith(3, null, @resourceFileContents)
@ResourceStateManager.checkProjectStateMatches("not-the-original-state", @basePath, @callback)
it "should call the callback with an error", ->
error = new Errors.FilesOutOfSyncError("invalid state for incremental update")
@callback.calledWith(error).should.equal true
describe "checkResourceFiles", ->
describe "when all the files are present", ->
beforeEach ->
@allFiles = [ @resources[0].path, @resources[1].path, @resources[2].path]
@ResourceStateManager.checkResourceFiles(@resources, @allFiles, @basePath, @callback)
it "should call the callback", ->
@callback.calledWithExactly().should.equal true
describe "when there is a missing file", ->
beforeEach ->
@allFiles = [ @resources[0].path, @resources[1].path]
@fs.stat = sinon.stub().callsArgWith(1, new Error())
@ResourceStateManager.checkResourceFiles(@resources, @allFiles, @basePath, @callback)
it "should call the callback with an error", ->
error = new Errors.FilesOutOfSyncError("resource files missing in incremental update")
@callback.calledWith(error).should.equal true
describe "when a resource contains a relative path", ->
beforeEach ->
@resources[0].path = "../foo/bar.tex"
@allFiles = [ @resources[0].path, @resources[1].path, @resources[2].path]
@ResourceStateManager.checkResourceFiles(@resources, @allFiles, @basePath, @callback)
it "should call the callback with an error", ->
@callback.calledWith(new Error("relative path in resource file list")).should.equal true

View File

@@ -1,17 +1,21 @@
SandboxedModule = require('sandboxed-module')
sinon = require('sinon')
require('chai').should()
should = require('chai').should()
modulePath = require('path').join __dirname, '../../../app/js/ResourceWriter'
path = require "path"
describe "ResourceWriter", ->
beforeEach ->
@ResourceWriter = SandboxedModule.require modulePath, requires:
"fs": @fs = {}
"fs": @fs =
mkdir: sinon.stub().callsArg(1)
unlink: sinon.stub().callsArg(1)
"./ResourceStateManager": @ResourceStateManager = {}
"wrench": @wrench = {}
"./UrlCache" : @UrlCache = {}
"mkdirp" : @mkdirp = sinon.stub().callsArg(1)
"./OutputFileFinder": @OutputFileFinder = {}
"logger-sharelatex": {log: sinon.stub(), err: sinon.stub()}
"./Metrics": @Metrics =
Timer: class Timer
done: sinon.stub()
@@ -19,7 +23,7 @@ describe "ResourceWriter", ->
@basePath = "/path/to/write/files/to"
@callback = sinon.stub()
describe "syncResourcesToDisk", ->
describe "syncResourcesToDisk on a full request", ->
beforeEach ->
@resources = [
"resource-1-mock"
@@ -28,7 +32,12 @@ describe "ResourceWriter", ->
]
@ResourceWriter._writeResourceToDisk = sinon.stub().callsArg(3)
@ResourceWriter._removeExtraneousFiles = sinon.stub().callsArg(2)
@ResourceWriter.syncResourcesToDisk(@project_id, @resources, @basePath, @callback)
@ResourceStateManager.saveProjectState = sinon.stub().callsArg(3)
@ResourceWriter.syncResourcesToDisk({
project_id: @project_id
syncState: @syncState = "0123456789abcdef"
resources: @resources
}, @basePath, @callback)
it "should remove old files", ->
@ResourceWriter._removeExtraneousFiles
@@ -41,9 +50,77 @@ describe "ResourceWriter", ->
.calledWith(@project_id, resource, @basePath)
.should.equal true
it "should store the sync state and resource list", ->
@ResourceStateManager.saveProjectState
.calledWith(@syncState, @resources, @basePath)
.should.equal true
it "should call the callback", ->
@callback.called.should.equal true
describe "syncResourcesToDisk on an incremental update", ->
beforeEach ->
@resources = [
"resource-1-mock"
]
@ResourceWriter._writeResourceToDisk = sinon.stub().callsArg(3)
@ResourceWriter._removeExtraneousFiles = sinon.stub().callsArgWith(2, null, @outputFiles = [], @allFiles = [])
@ResourceStateManager.checkProjectStateMatches = sinon.stub().callsArgWith(2, null, @resources)
@ResourceStateManager.saveProjectState = sinon.stub().callsArg(3)
@ResourceStateManager.checkResourceFiles = sinon.stub().callsArg(3)
@ResourceWriter.syncResourcesToDisk({
project_id: @project_id,
syncType: "incremental",
syncState: @syncState = "1234567890abcdef",
resources: @resources
}, @basePath, @callback)
it "should check the sync state matches", ->
@ResourceStateManager.checkProjectStateMatches
.calledWith(@syncState, @basePath)
.should.equal true
it "should remove old files", ->
@ResourceWriter._removeExtraneousFiles
.calledWith(@resources, @basePath)
.should.equal true
it "should check each resource exists", ->
@ResourceStateManager.checkResourceFiles
.calledWith(@resources, @allFiles, @basePath)
.should.equal true
it "should write each resource to disk", ->
for resource in @resources
@ResourceWriter._writeResourceToDisk
.calledWith(@project_id, resource, @basePath)
.should.equal true
it "should call the callback", ->
@callback.called.should.equal true
describe "syncResourcesToDisk on an incremental update when the state does not match", ->
beforeEach ->
@resources = [
"resource-1-mock"
]
@ResourceStateManager.checkProjectStateMatches = sinon.stub().callsArgWith(2, @error = new Error())
@ResourceWriter.syncResourcesToDisk({
project_id: @project_id,
syncType: "incremental",
syncState: @syncState = "1234567890abcdef",
resources: @resources
}, @basePath, @callback)
it "should check whether the sync state matches", ->
@ResourceStateManager.checkProjectStateMatches
.calledWith(@syncState, @basePath)
.should.equal true
it "should call the callback with an error", ->
@callback.calledWith(@error).should.equal true
describe "_removeExtraneousFiles", ->
beforeEach ->
@output_files = [{
@@ -55,6 +132,8 @@ describe "ResourceWriter", ->
}, {
path: "extra.aux"
type: "aux"
}, {
path: "cache/_chunk1"
}]
@resources = "mock-resources"
@OutputFileFinder.findOutputFiles = sinon.stub().callsArgWith(2, null, @output_files)
@@ -80,6 +159,11 @@ describe "ResourceWriter", ->
@ResourceWriter._deleteFileIfNotDirectory
.calledWith(path.join(@basePath, "extra.aux"))
.should.equal false
it "should not delete the knitr cache file", ->
@ResourceWriter._deleteFileIfNotDirectory
.calledWith(path.join(@basePath, "cache/_chunk1"))
.should.equal false
it "should call the callback", ->
@callback.called.should.equal true
@@ -94,7 +178,7 @@ describe "ResourceWriter", ->
path: "main.tex"
url: "http://www.example.com/main.tex"
modified: Date.now()
@UrlCache.downloadUrlToFile = sinon.stub().callsArg(4)
@UrlCache.downloadUrlToFile = sinon.stub().callsArgWith(4, "fake error downloading file")
@ResourceWriter._writeResourceToDisk(@project_id, @resource, @basePath, @callback)
it "should ensure the directory exists", ->
@@ -110,6 +194,9 @@ describe "ResourceWriter", ->
it "should call the callback", ->
@callback.called.should.equal true
it "should not return an error if the resource writer errored", ->
should.not.exist @callback.args[0][0]
describe "with a content based resource", ->
beforeEach ->
@resource =
@@ -147,6 +234,27 @@ describe "ResourceWriter", ->
.calledWith(new Error("resource path is outside root directory"))
.should.equal true
describe "checkPath", ->
describe "with a valid path", ->
beforeEach ->
@ResourceWriter.checkPath("foo", "bar", @callback)
it "should return the joined path", ->
@callback.calledWith(null, "foo/bar")
.should.equal true
describe "with an invalid path", ->
beforeEach ->
@ResourceWriter.checkPath("foo", "baz/../../bar", @callback)
it "should return an error", ->
@callback.calledWith(new Error("resource path is outside root directory"))
.should.equal true
describe "with another invalid path matching on a prefix", ->
beforeEach ->
@ResourceWriter.checkPath("foo", "../foobar/baz", @callback)
it "should return an error", ->
@callback.calledWith(new Error("resource path is outside root directory"))
.should.equal true

View File

@@ -0,0 +1,158 @@
should = require('chai').should()
SandboxedModule = require('sandboxed-module')
assert = require('assert')
path = require('path')
sinon = require('sinon')
modulePath = path.join __dirname, "../../../app/js/StaticServerForbidSymlinks"
expect = require("chai").expect
describe "StaticServerForbidSymlinks", ->
beforeEach ->
@settings =
path:
compilesDir: "/compiles/here"
@fs = {}
@ForbidSymlinks = SandboxedModule.require modulePath, requires:
"settings-sharelatex":@settings
"logger-sharelatex":
log:->
warn:->
error:->
"fs":@fs
@dummyStatic = (rootDir, options) ->
return (req, res, next) ->
# console.log "dummyStatic serving file", rootDir, "called with", req.url
# serve it
next()
@StaticServerForbidSymlinks = @ForbidSymlinks @dummyStatic, @settings.path.compilesDir
@req =
params:
project_id:"12345"
@res = {}
@req.url = "/12345/output.pdf"
describe "sending a normal file through", ->
beforeEach ->
@fs.realpath = sinon.stub().callsArgWith(1, null, "#{@settings.path.compilesDir}/#{@req.params.project_id}/output.pdf")
it "should call next", (done)->
@res.sendStatus = (resCode)->
resCode.should.equal 200
done()
@StaticServerForbidSymlinks @req, @res, done
describe "with a missing file", ->
beforeEach ->
@fs.realpath = sinon.stub().callsArgWith(1, {code: 'ENOENT'}, "#{@settings.path.compilesDir}/#{@req.params.project_id}/unknown.pdf")
it "should send a 404", (done)->
@res.sendStatus = (resCode)->
resCode.should.equal 404
done()
@StaticServerForbidSymlinks @req, @res
describe "with a symlink file", ->
beforeEach ->
@fs.realpath = sinon.stub().callsArgWith(1, null, "/etc/#{@req.params.project_id}/output.pdf")
it "should send a 404", (done)->
@res.sendStatus = (resCode)->
resCode.should.equal 404
done()
@StaticServerForbidSymlinks @req, @res
describe "with a relative file", ->
beforeEach ->
@req.url = "/12345/../67890/output.pdf"
it "should send a 404", (done)->
@res.sendStatus = (resCode)->
resCode.should.equal 404
done()
@StaticServerForbidSymlinks @req, @res
describe "with a unnormalized file containing .", ->
beforeEach ->
@req.url = "/12345/foo/./output.pdf"
it "should send a 404", (done)->
@res.sendStatus = (resCode)->
resCode.should.equal 404
done()
@StaticServerForbidSymlinks @req, @res
describe "with a file containing an empty path", ->
beforeEach ->
@req.url = "/12345/foo//output.pdf"
it "should send a 404", (done)->
@res.sendStatus = (resCode)->
resCode.should.equal 404
done()
@StaticServerForbidSymlinks @req, @res
describe "with a non-project file", ->
beforeEach ->
@req.url = "/.foo/output.pdf"
it "should send a 404", (done)->
@res.sendStatus = (resCode)->
resCode.should.equal 404
done()
@StaticServerForbidSymlinks @req, @res
describe "with a file outside the compiledir", ->
beforeEach ->
@req.url = "/../bar/output.pdf"
it "should send a 404", (done)->
@res.sendStatus = (resCode)->
resCode.should.equal 404
done()
@StaticServerForbidSymlinks @req, @res
describe "with a file with no leading /", ->
beforeEach ->
@req.url = "./../bar/output.pdf"
it "should send a 404", (done)->
@res.sendStatus = (resCode)->
resCode.should.equal 404
done()
@StaticServerForbidSymlinks @req, @res
describe "with a github style path", ->
beforeEach ->
@req.url = "/henryoswald-latex_example/output/output.log"
@fs.realpath = sinon.stub().callsArgWith(1, null, "#{@settings.path.compilesDir}/henryoswald-latex_example/output/output.log")
it "should call next", (done)->
@res.sendStatus = (resCode)->
resCode.should.equal 200
done()
@StaticServerForbidSymlinks @req, @res, done
describe "with an error from fs.realpath", ->
beforeEach ->
@fs.realpath = sinon.stub().callsArgWith(1, "error")
it "should send a 500", (done)->
@res.sendStatus = (resCode)->
resCode.should.equal 500
done()
@StaticServerForbidSymlinks @req, @res

View File

@@ -0,0 +1,101 @@
SandboxedModule = require('sandboxed-module')
sinon = require('sinon')
require('chai').should()
modulePath = require('path').join __dirname, '../../../app/js/TikzManager'
describe 'TikzManager', ->
beforeEach ->
@TikzManager = SandboxedModule.require modulePath, requires:
"./ResourceWriter": @ResourceWriter = {}
"./SafeReader": @SafeReader = {}
"fs": @fs = {}
"logger-sharelatex": @logger = {log: () ->}
describe "checkMainFile", ->
beforeEach ->
@compileDir = "compile-dir"
@mainFile = "main.tex"
@callback = sinon.stub()
describe "if there is already an output.tex file in the resources", ->
beforeEach ->
@resources = [{path:"main.tex"},{path:"output.tex"}]
@TikzManager.checkMainFile @compileDir, @mainFile, @resources, @callback
it "should call the callback with false ", ->
@callback.calledWithExactly(null, false)
.should.equal true
describe "if there is no output.tex file in the resources", ->
beforeEach ->
@resources = [{path:"main.tex"}]
@ResourceWriter.checkPath = sinon.stub()
.withArgs(@compileDir, @mainFile)
.callsArgWith(2, null, "#{@compileDir}/#{@mainFile}")
describe "and the main file contains tikzexternalize", ->
beforeEach ->
@SafeReader.readFile = sinon.stub()
.withArgs("#{@compileDir}/#{@mainFile}")
.callsArgWith(3, null, "hello \\tikzexternalize")
@TikzManager.checkMainFile @compileDir, @mainFile, @resources, @callback
it "should look at the file on disk", ->
@SafeReader.readFile
.calledWith("#{@compileDir}/#{@mainFile}")
.should.equal true
it "should call the callback with true ", ->
@callback.calledWithExactly(null, true)
.should.equal true
describe "and the main file does not contain tikzexternalize", ->
beforeEach ->
@SafeReader.readFile = sinon.stub()
.withArgs("#{@compileDir}/#{@mainFile}")
.callsArgWith(3, null, "hello")
@TikzManager.checkMainFile @compileDir, @mainFile, @resources, @callback
it "should look at the file on disk", ->
@SafeReader.readFile
.calledWith("#{@compileDir}/#{@mainFile}")
.should.equal true
it "should call the callback with false", ->
@callback.calledWithExactly(null, false)
.should.equal true
describe "injectOutputFile", ->
beforeEach ->
@rootDir = "/mock"
@filename = "filename.tex"
@callback = sinon.stub()
@content = '''
\\documentclass{article}
\\usepackage{tikz}
\\tikzexternalize
\\begin{document}
Hello world
\\end{document}
'''
@fs.readFile = sinon.stub().callsArgWith(2, null, @content)
@fs.writeFile = sinon.stub().callsArg(3)
@ResourceWriter.checkPath = sinon.stub().callsArgWith(2, null, "#{@rootDir}/#{@filename}")
@TikzManager.injectOutputFile @rootDir, @filename, @callback
it "sould check the path", ->
@ResourceWriter.checkPath.calledWith(@rootDir, @filename)
.should.equal true
it "should read the file", ->
@fs.readFile
.calledWith("#{@rootDir}/#{@filename}", "utf8")
.should.equal true
it "should write out the same file as output.tex", ->
@fs.writeFile
.calledWith("#{@rootDir}/output.tex", @content, {flag: 'wx'})
.should.equal true
it "should call the callback", ->
@callback.called.should.equal true

View File

@@ -11,6 +11,7 @@ describe "UrlFetcher", ->
@UrlFetcher = SandboxedModule.require modulePath, requires:
request: defaults: @defaults = sinon.stub().returns(@request = {})
fs: @fs = {}
"logger-sharelatex": @logger = { log: sinon.stub(), error: sinon.stub() }
it "should turn off the cookie jar in request", ->
@defaults.calledWith(jar: false)
@@ -21,24 +22,29 @@ describe "UrlFetcher", ->
@path = "/path/to/file/on/disk"
@request.get = sinon.stub().returns(@urlStream = new EventEmitter)
@urlStream.pipe = sinon.stub()
@fs.createWriteStream = sinon.stub().returns(@fileStream = "write-stream-stub")
@urlStream.pause = sinon.stub()
@urlStream.resume = sinon.stub()
@fs.createWriteStream = sinon.stub().returns(@fileStream = new EventEmitter)
@fs.unlink = (file, callback) -> callback()
@UrlFetcher.pipeUrlToFile(@url, @path, @callback)
it "should request the URL", ->
@request.get
.calledWith(@url)
.calledWith(sinon.match {"url": @url})
.should.equal true
it "should open the file for writing", ->
@fs.createWriteStream
.calledWith(@path)
.should.equal true
describe "successfully", ->
beforeEach ->
@res = statusCode: 200
@urlStream.emit "response", @res
@urlStream.emit "end"
@fileStream.emit "finish"
it "should open the file for writing", ->
@fs.createWriteStream
.calledWith(@path)
.should.equal true
it "should pipe the URL to the file", ->
@urlStream.pipe