As part of improving our test automation, we decided it would be useful to enable Visual Studio to build Cordova projects without access to the internet. We’ve decided to share the fruits of that effort in the hopes it might be useful to others. The following is a guide on how to create an environment that allows limited offline development of Cordova projects in Visual Studio.
Build offline
In order to create an offline build environment, we need to take resources from an internet connected machine, pack them up, and deploy them to a target machine. There are a few things we need:
The NPM cache
The Cordova cache, with the platforms we intend to build against
The sandboxed versions of Node/NPM that VS uses
At the end of the blog post, there’s a project with a few scripts that show how most of the process can be automated.
Gather resources from a source machine
First, we should ensure we have NPM installed on our source machine. And, to ensure we take only what we need, we should start with a clean NPM cache.
npm cache clean
Once we’ve done that, we should globally install Cordova, along with any plugins our project is using. We’ll also need the whitelist plugin.
There’s also an NPM package our team wrote to help with the build process. This plugin will already exist on our target machine. However, its dependencies will not. So, we should globally install it so our NPM cache contains its dependencies.
npm install -g C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\ApacheCordovaTools\Packages\vs-tac
Next, we should install the packages containing the platforms we want to build against. Each Cordova version has a specific set of pinned platform versions. We’ll need exactly these versions of the platform plugins, otherwise VS might try to access the internet to download it. Cordova CLI usually lists the pinned versions of the platforms in their update changelogs. However, if we have trouble finding these lists, we could instead create a Cordova project using the version of the Cordova CLI we want, then add the platforms we’re using. Doing so should give us the pinned version of those platforms.
npm install -g cordova-windows@4.3.1
The platform packages are actually used to create the Cordova cache on the target machine. We could try to copy our cordova cache manually, but this has almost always exceeded the Windows max path character limit. We’ll download this package and extract it to the Cordova cache on the target machine instead.
At this point, we should copy our entire NPM cache folder (usually %AppData%\npm-cache) to a temporary directory. We should also copy %AppData%\npm\node_modules to our temporary directory. With these, we have everything we need to populate our NPM and Cordova caches. However, we still need the sandboxed versions of NPM and Node that Visual Studio uses. If we’ve run a successful build, we should be able to find these files at %AppData%\Microsoft\VisualStudio\MDA\vs-npm\2.15.1. We should copy this to our temporary directory.
In certain cases, this directory may also exceed the Windows max path character limit. If that happens, we can download the parts individually and copy them to our temporary directory.
Also copy nodevars.bat from the root of our Node installation, or from the sandboxed node/NPM installation.
Our temporary directory now contains everything a target machine needs to build offline. Archiving the files isn’t strictly necessary, though doing so aids in transferring the resources to a target machine.
Deploying to a target machine
This is mostly the opposite of what we’ve done, with a few exceptions if we hit max path issues.
First, extract the NPM cache to %AppData%\npm-cache. Extract node_modules to %AppData%\npm\node_modules
Then, we need to un-tar the platform packages to the Cordova cache. The contents of that platform’s .\package.tgz\package.tar should be extracted to %HomePath%\.cordova\lib\[platform name]\cordova\[platform version]. For example, the Cordova Windows platform we downloaded earlier would be extracted to %HomePath%\.cordova\lib\windows\cordova\4.3.1.
Finally, we need to sandbox NPM and Node. If we were able to copy the entire sandboxed directory, we just extract it to the same directory we got it from. If we downloaded the parts individually, copy the files to %AppData%\Microsoft\VisualStudio\MDA\vs-npm\2.15.1 and establish this directory structure.
.\node.exe
.\nodevars.bat
.\npm
.\npm.cmd
.\node_modules\npm[contents of v2.15.1.zip]
If everything is in place, VS shouldn’t need to access the internet during a build, unless we specifically invoke a feature that requires the internet (adding a new plugin, for example).
Automation
This is all pretty annoying to do every time. We wrote a couple scripts for our test automation that gather, archive, and extract these dependencies. Below is a somewhat simplified version of these scripts. With some tweaking, you may be able to get this to work for your project. But mostly, it should serve as an example of how you might accomplish automating the creation of these offline build archives.
If you want to use the scripts directly, you’ll need 7zip. We use it to handle a few cases where max path might be an issue, and also to un-tar the platforms package.
Building the project will output a zip file and script that can be run on a target machine. The script will install the contents of the archive to the correct places on the target machine. Feel free to modify it to fit your needs!
We recently had a customer reach out to us to help him fix a problem deploying to a local iOS device.
Here’s a snippet from his build error:
2>Found iTunes version: 12.4.1.6
2>Uninstalling app ID: io.cordova.myapp439183.
2>Successfully installed .ipa file: C:\Users\xxxxxx\documents\visual studio 2015\Projects\BlankCordovaApp2\BlankCordovaApp2\bin\iOS\Debug\BlankCordovaApp2.ipa.
2>Starting idevicedebugserverproxy.exe at port 3000.
2>Timed out connecting to local iOS device.
2>Failed to launch the app on the device. Check that the device is connected, awake, and unlocked.
2>Deployment failed.
Upon investigation, we found the problem was due to a mismatch between the iOS version of his device and the iOS device support folder on his Mac. The solution
was to update XCode to 7.3.1. This updated the device support folder to a version that worked with iOS 9.3.1.
If you run into a deployment error like the one above, updating XCode should fix the problem.
Deprecated support for plugin add prior to Cordova 5.0.0
Cordova has discontinued support for plugin downloads through Cordova versions prior to 5.0.0. We’ve updated our tools to handle this gracefully by providing a warning and preventing users from adding plugins to deprecated versions of Cordova.
Core plugin parameter support
With this update, we’ve added a new plugin to the core plugin list that requires parameters, and so we’ve extended our plugin parameter support to include the core plugin list.
Core Plugin Changes
We’ve updated the core plugin list with new plugins.
Added Cordova SQLite plugin
Added Windows-extended version of Cordova SQLite plugin
Added Microsoft Intune App SDK plugin
Removed Cordova WebSQL plugin
Bug Fixes
Missing .appxupload file for Windows 10 Store Packaging
We fixed a bug where store packaging for Windows 10 wouldn’t generate an .appxupload file. Certain properties required by the packaging system were missing from the Windows 10 Cordova jsproj file, so we updated the build process to include them.
These new properties remain in your project after the build completes, but are removed the next time you build within VS. Please keep this in mind if you like to combine IDE and CLI development workflows.
Some plugins cannot be added by ID
Due to a bug in how we detect and add Cordova plugins, certain plugins could not be added by ID. This has been fixed.
Accessibility bugs
We’ve fixed a few accessibility bugs.
Other crashes
We fixed a number of crashes that were reported to us from our crash reporting system.
Feedback and Thanks
We want to thank you all again for your comments and feedback!
If you try our update and run into issues, we want to know about it. If you run into any problems with the update, send me an email so we can figure it out.
The ASP.NET team recently released Microsoft ASP.NET Web Tools Preview 1 RC2 which include tooling changes of interest to developers working with Cordova or ASP.NET. Specifically, they’ve updated the version of Node and NPM used by Visual Studio 2015 for NPM package restore and the Task Runner Explorer.
Prior to this update, newer versions of NPM cause some interoperability problems between the Visual Studio Package Manager and the command line, that could affect your Cordova development workflow.
Note: if you don’t want to install the new preview release, you can still configure Visual Studio to use other version of node/npm as described here, however it has not been optimized for NPM 3.x.
Here’s a summary of the two major issues fixed by this preview release, which I think are of interest to Visual Studio TACO developers.
Extraneous Dependencies
Recent versions of node install recent versions of npm, which use a flat folder (not deeply nested) structure for node_modules that was introduced in npm 3. However, when the same project was opened in Visual Studio TACO it would try to restore using an older sandboxed version of npm. This caused Visual Studio to incorrectly show dependencies as “extraneous”.
Task Runner failed to load Gulp tasks
Another common problem you may have seen is an error similar to the following, in the Output Window under the Task runner explorer logs:
Failed to run "C:\code\TacoApp\Gulpfile.js"...
cmd.exe /c gulp --tasks-simple
Error: Missing binding C:\code\TacoApp\node_modules\node-sass\vendor\win32-ia32-11\binding.node
Node Sass could not find a binding for your current environment: Windows 32-bit with Node 0.10.x
Found bindings for the following environments:
- Windows 64-bit with Node.js 4.x
This usually happens because your environment has changed since running npm install.
This is caused when node_modules were restored from the command line using a different version of Node than what Visual Studio uses. With this update, Visual Studio is using a much newer version of Node and NPM, which will make this error message much less common.
Note: You may still run into this issue in the future. The easiest way to fix it is removing the node_modules folder, and let Visual Studio restore them again: Open the solution explorer, find the Dependencies node, right-click and select Restore Packages.
Do you use Visual Studio TACO to build iOS applications? If so, I’d like to talk with you to learn more about what your experience has been like! Please, email me: jomatthi -at- microsoft -dot- com, or message me on Twitter and I’ll follow you back to chat (just make it clear why you’re messaging me).
In the TACO team we talk with developers every day and we learn a lot from these conversations; sometimes a simple chat can reveal whole new problems and challenges that we could help ease through tooling. Right now, the VS TACO iOS build experience is my main area of focus and I look forward to hearing your feedback about this!
Let’s talk, soon.
Jordan Matthiesen
Program Manager | Tools for Apache Cordova (TACO)
jomatthi -at- microsoft -dot- com | @JMatthiesen
