CRLF vs LF mismatches errors when patching OpenCollada in "make deps" #74018

New Issue

Julien Rivaud · 2020-02-19T21:52:43+01:00

Julien Rivaud commented

2020-02-19 21:52:43 +01:00

When I recently tried to "make deps" as a prerequisite to "make full", I got a failure at «external_opencollada» where the patching phase failed due to dos/unix line endings mismatches (CRLF vs LF).
I had to modify the build_files/build_environment/patches/opencollada.diff file to prepend CRs for CMakeLists.txt and Externals/LibXML/CMakeLists.txt
And change "opencollada.cmake" to pass the --binary option to "patch"

Last time I built a static blender from scratch, the problem didn't exist. I tried destroying the complete build_linux and build_linux_full dirs, and that didn't solve anything. Is the problem known ? How often do you rebuild the deps in the buildbots ?

I use the CentOs7 reference platform, with the same script as what sergey uses to setup blender's buildbots. I am trying to build f6d5a95513 which is a recent master HEAD.

Here is the diff I had to apply for "make deps" to succeed, where ^M denotes a CR char.

diff --git a/build_files/build_environment/cmake/opencollada.cmake b/build_files/build_environment/cmake/opencollada.cmake
index e1a8abc40ce..d798413c5e8 100644
--- a/build_files/build_environment/cmake/opencollada.cmake
+++ b/build_files/build_environment/cmake/opencollada.cmake
@@ -27,7 +27,7 @@ ExternalProject_Add(external_opencollada
   DOWNLOAD_DIR ${DOWNLOAD_DIR}
   URL_HASH MD5=${OPENCOLLADA_HASH}
   PREFIX ${BUILD_DIR}/opencollada
-  PATCH_COMMAND ${PATCH_CMD} -p 1 -N -d ${BUILD_DIR}/opencollada/src/external_opencollada < ${PATCH_DIR}/opencollada.diff
+  PATCH_COMMAND ${PATCH_CMD} --binary -l -p 1 -N -d ${BUILD_DIR}/opencollada/src/external_opencollada < ${PATCH_DIR}/opencollada.diff
   CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${LIBDIR}/opencollada ${DEFAULT_CMAKE_FLAGS} ${OPENCOLLADA_EXTRA_ARGS}
   INSTALL_DIR ${LIBDIR}/opencollada
 )
diff --git a/build_files/build_environment/patches/opencollada.diff b/build_files/build_environment/patches/opencollada.diff
index e7f563acaa1..387f9d1ccbd 100644
--- a/build_files/build_environment/patches/opencollada.diff
+++ b/build_files/build_environment/patches/opencollada.diff
@@ -3,19 +3,19 @@ index 95abbe2..4f14f30 100644
 --- a/CMakeLists.txt
 +++ b/CMakeLists.txt
 @@ -254,11 +254,11 @@ if(USE_STATIC_MSVC_RUNTIME)
- endif()
- 
- #adding PCRE
--find_package(PCRE)
-+#find_package(PCRE)
- if (PCRE_FOUND)
- 	message(STATUS "SUCCESSFUL: PCRE found")
- else ()  # if pcre not found building its local copy from ./Externals
--	if (WIN32 OR APPLE)
-+	if (1)
- 		message("WARNING: Native PCRE not found, taking PCRE from ./Externals")
- 		add_definitions(-DPCRE_STATIC)
- 		add_subdirectory(${EXTERNAL_LIBRARIES}/pcre)
+ endif()^M
+ ^M
+ #adding PCRE^M
+-find_package(PCRE)^M
++#find_package(PCRE)^M
+ if (PCRE_FOUND)^M
+ 	message(STATUS "SUCCESSFUL: PCRE found")^M
+ else ()  # if pcre not found building its local copy from ./Externals^M
+-	if (WIN32 OR APPLE)^M
++	if (1)^M
+ 		message("WARNING: Native PCRE not found, taking PCRE from ./Externals")^M
+ 		add_definitions(-DPCRE_STATIC)^M
+ 		add_subdirectory(${EXTERNAL_LIBRARIES}/pcre)^M
 diff --git a/DAEValidator/library/include/no_warning_begin b/DAEValidator/library/include/no_warning_begin
 index 7a69c32..defb315 100644
 --- a/DAEValidator/library/include/no_warning_begin
@@ -78,10 +78,10 @@ diff -Naur orig/Externals/LibXML/CMakeLists.txt external_opencollada/Externals/L
 --- orig/Externals/LibXML/CMakeLists.txt	2018-11-26 15:43:10 -0700
 +++ external_opencollada/Externals/LibXML/CMakeLists.txt	2020-01-21 14:15:04 -0700
 @@ -9,6 +9,7 @@
- 		-DLIBXML_SCHEMAS_ENABLED
- 		-DLIBXML_XPATH_ENABLED
- 		-DLIBXML_TREE_ENABLED
-+		-DLIBXML_STATIC
- 	)
- 	
- if(USE_STATIC_MSVC_RUNTIME)
+ 		-DLIBXML_SCHEMAS_ENABLED^M
+ 		-DLIBXML_XPATH_ENABLED^M
+ 		-DLIBXML_TREE_ENABLED^M
++		-DLIBXML_STATIC^M
+ 	)^M
+ 	^M
+ if(USE_STATIC_MSVC_RUNTIME)^M

When I recently tried to "make deps" as a prerequisite to "make full", I got a failure at «external_opencollada» where the patching phase failed due to dos/unix line endings mismatches (CRLF vs LF). I had to modify the build_files/build_environment/patches/opencollada.diff file to prepend CRs for CMakeLists.txt and Externals/LibXML/CMakeLists.txt And change "opencollada.cmake" to pass the --binary option to "patch" Last time I built a static blender from scratch, the problem didn't exist. I tried destroying the complete build_linux and build_linux_full dirs, and that didn't solve anything. Is the problem known ? How often do you rebuild the deps in the buildbots ? I use the CentOs7 reference platform, with the same script as what sergey uses to setup blender's buildbots. I am trying to build f6d5a9551380581167 which is a recent master HEAD. Here is the diff I had to apply for "make deps" to succeed, where ^M denotes a CR char. ``` diff --git a/build_files/build_environment/cmake/opencollada.cmake b/build_files/build_environment/cmake/opencollada.cmake index e1a8abc40ce..d798413c5e8 100644 --- a/build_files/build_environment/cmake/opencollada.cmake +++ b/build_files/build_environment/cmake/opencollada.cmake @@ -27,7 +27,7 @@ ExternalProject_Add(external_opencollada DOWNLOAD_DIR ${DOWNLOAD_DIR} URL_HASH MD5=${OPENCOLLADA_HASH} PREFIX ${BUILD_DIR}/opencollada - PATCH_COMMAND ${PATCH_CMD} -p 1 -N -d ${BUILD_DIR}/opencollada/src/external_opencollada < ${PATCH_DIR}/opencollada.diff + PATCH_COMMAND ${PATCH_CMD} --binary -l -p 1 -N -d ${BUILD_DIR}/opencollada/src/external_opencollada < ${PATCH_DIR}/opencollada.diff CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${LIBDIR}/opencollada ${DEFAULT_CMAKE_FLAGS} ${OPENCOLLADA_EXTRA_ARGS} INSTALL_DIR ${LIBDIR}/opencollada ) diff --git a/build_files/build_environment/patches/opencollada.diff b/build_files/build_environment/patches/opencollada.diff index e7f563acaa1..387f9d1ccbd 100644 --- a/build_files/build_environment/patches/opencollada.diff +++ b/build_files/build_environment/patches/opencollada.diff @@ -3,19 +3,19 @@ index 95abbe2..4f14f30 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -254,11 +254,11 @@ if(USE_STATIC_MSVC_RUNTIME) - endif() - - #adding PCRE --find_package(PCRE) -+#find_package(PCRE) - if (PCRE_FOUND) - message(STATUS "SUCCESSFUL: PCRE found") - else () # if pcre not found building its local copy from ./Externals -- if (WIN32 OR APPLE) -+ if (1) - message("WARNING: Native PCRE not found, taking PCRE from ./Externals") - add_definitions(-DPCRE_STATIC) - add_subdirectory(${EXTERNAL_LIBRARIES}/pcre) + endif()^M + ^M + #adding PCRE^M +-find_package(PCRE)^M ++#find_package(PCRE)^M + if (PCRE_FOUND)^M + message(STATUS "SUCCESSFUL: PCRE found")^M + else () # if pcre not found building its local copy from ./Externals^M +- if (WIN32 OR APPLE)^M ++ if (1)^M + message("WARNING: Native PCRE not found, taking PCRE from ./Externals")^M + add_definitions(-DPCRE_STATIC)^M + add_subdirectory(${EXTERNAL_LIBRARIES}/pcre)^M diff --git a/DAEValidator/library/include/no_warning_begin b/DAEValidator/library/include/no_warning_begin index 7a69c32..defb315 100644 --- a/DAEValidator/library/include/no_warning_begin @@ -78,10 +78,10 @@ diff -Naur orig/Externals/LibXML/CMakeLists.txt external_opencollada/Externals/L --- orig/Externals/LibXML/CMakeLists.txt 2018-11-26 15:43:10 -0700 +++ external_opencollada/Externals/LibXML/CMakeLists.txt 2020-01-21 14:15:04 -0700 @@ -9,6 +9,7 @@ - -DLIBXML_SCHEMAS_ENABLED - -DLIBXML_XPATH_ENABLED - -DLIBXML_TREE_ENABLED -+ -DLIBXML_STATIC - ) - - if(USE_STATIC_MSVC_RUNTIME) + -DLIBXML_SCHEMAS_ENABLED^M + -DLIBXML_XPATH_ENABLED^M + -DLIBXML_TREE_ENABLED^M ++ -DLIBXML_STATIC^M + )^M + ^M + if(USE_STATIC_MSVC_RUNTIME)^M ```

Julien Rivaud commented

2020-02-19 21:52:43 +01:00

Added subscriber: @FrnchFrgg

Richard Antalik commented

2020-02-21 12:58:06 +01:00

Added subscribers: @Sergey, @iss

Richard Antalik commented

2020-02-21 12:58:06 +01:00

I am really not sure here. I tried to apply patch and it failed with "patch does not apply"

I would suggest asking on https://blender.chat or @Sergey directly

I am really not sure here. I tried to apply patch and it failed with "patch does not apply" I would suggest asking on https://blender.chat or @Sergey directly

Julien Rivaud commented

2020-02-21 13:11:17 +01:00

The patch is not applicable as-is, since I had to edit it to replace CRs by ^M so that you can see them. Doing so or posting the patch here might have corrupted it by removing or adding a space, and since it is a diff of diffs formatting issues will confuse git or patch.

Julien Rivaud commented

2020-02-21 13:13:25 +01:00

I also tried to ask on #blender in freenode, but had no answer. Such highly specific problems are not suited to a user IRC channel, IMHO.

Sergey Sharybin commented

2020-02-21 16:34:27 +01:00

Added subscriber: @LazyDodo

Sergey Sharybin commented

2020-02-21 16:34:27 +01:00

This is something what I've fixed in c2a251d655 but it was re-introduced in 8c509f98e1.

Would really be nice to find a solution which will not be so much fragile. Maybe convert files to native EOL? @LazyDodo, any other ideas?

This is something what I've fixed in c2a251d655e but it was re-introduced in 8c509f98e12. Would really be nice to find a solution which will not be so much fragile. Maybe convert files to native EOL? @LazyDodo, any other ideas?

Aaron Carlisle commented

2020-02-28 04:17:05 +01:00

Added subscriber: @Blendify

Aaron Carlisle commented

2020-02-28 04:17:05 +01:00

Changed status from 'Needs Triage' to: 'Resolved'

Aaron Carlisle closed this issue

2020-02-28 04:17:05 +01:00

Aaron Carlisle self-assigned this 2020-02-28 04:17:05 +01:00

Aaron Carlisle commented

2020-02-28 04:17:05 +01:00

This was fixed by d109ea5ee5

This was fixed by d109ea5ee5

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

CRLF vs LF mismatches errors when patching OpenCollada in "make deps" #74018