Dactyloidae iOS initial commit

mobile/ios/Docs/BUILDING.md

@@ -0,0 +1,118 @@
+Building Firefox for iOS
+========================
+
+Prerequisites, as of *February 14, 2017*:
+
+* Mac OS X 10.11.5
+* Xcode 8.2.1 with the iOS 10 SDK (Betas not supported)
+* Carthage 0.15 or newer
+
+When running on a device:
+
+* A device that supports iOS 9.3 GM or later
+* One of the following:
+ * A developer account and Admin access to the *Certificates, Identifiers & Profiles* section of the *iOS DevCenter*
+ * A free developer account (create an Apple ID for free and add as an account in Xcode)
+
+Get the Code
+-----------
+
+```
+git clone https://github.com/mozilla/firefox-ios
+cd firefox-ios
+```
+
+(If you have forked the repository, substitute the URL with your own repository location.)
+
+Pull in Dependencies
+--------------------
+
+We use Carthage to manage projects that we depend on. __The build will currently only work with Carthage v0.15 or newer__. If you do not already have Carthage installed, you need to grab it via Homebrew. Assuming you have Homebrew installed, execute the following:
+
+```
+brew update
+brew upgrade
+brew install carthage
+```
+
+You can now execute our `bootstrap.sh` script:
+
+```
+./bootstrap.sh
+```
+
+At this point you have checked out the source code for both the Firefox for iOS project and built it's dependencies. You can now build and run the application.
+
+Everything after this point is done from within Xcode.
+
+Run on the Simulator
+-----------------
+
+* Open `Client.xcodeproj` and make sure you have the *Fennec* scheme and a simulated device selected. The app should run on any simulator. We just have not tested very well on the *Resizable iPad* and *Resizable iPhone* simulators.
+* Select *Product -> Run* and the application should build and run on the selected simulator.
+
+Run on a Device with Xcode 8 and a Free Developer Account
+---------------
+
+> Only follow these instructions if you are using the free personal developer accounts. Simply add your Apple ID as an account in Xcode.
+
+Since the bundle identifier we use for Firefox is tied to our developer account, you'll need to generate your own identifier and update the existing configuration.
+
+1. Open Client/Configuration/Fennec.xcconfig
+2. Change MOZ_BUNDLE_ID to your own bundle identifier. Just think of something unique: e.g., com.your_github_id.Fennec
+3. Open the project editor in Xcode.
+4. For the 'Client' target, in the 'Capabilities' section, turn off the capabilities 'Push Notifications' and 'Wallet'.
+5. For each target, in the 'General' section, under 'Signing', select your personal development account.
+
+If you submit a patch, be sure to exclude these files because they are only relevant for your personal build.
+
+> If after building, Xcode fails to run the app with a vague `Security` error, open Settings -> Profiles on your iOS Device and Trust your personal developer profile. This may only happen on iOS 9.
+
+Run on a Device
+---------------
+
+These are instructions for development. Not production / distribution.
+
+> Before you try to run the application on a device, it is highly recommended that you first make sure that you can run applications on device in general. Just create one of the built-in iOS templates that Xcode provides and make sure you can run that on your device. If you can then it means you have done the basic setup like pairing your device, registering its UDID in the dev center, etc.
+
+Before you can run the application on your device, you need to setup a few things in the *Certificates, Identifiers & Profiles* section of the iOS Developer Center.
+
+> _Note_: When we mention `YOURREVERSEDOMAIN` below, use your own domain in reverse notation like `com.example` or if you do not have your own domain, just use something unique and personal like `io.github.yourgithubusername`. Please do not use existing domain names which you do not own.
+
+1. Create a Application Group. Name this group 'Fennec' and for its Identifier use `group.YOURREVERSEDOMAIN.Fennec`
+2. Create a new App Id. Name it 'Fennec'. Give it an Explicit App ID and set its Bundle Identifier to `YOURREVERSEDOMAIN.Fennec`. In the App Services section, select *App Groups*.
+3. Create a new App Id. Name it 'Fennec ShareTo'. Give it an Explicit App ID and set its Bundle Identifier to `YOURREVERSEDOMAIN.Fennec.ShareTo`. In the App Services section, select *App Groups*.
+4. Create a new App Id. Name it 'Fennec SendTo'. Give it an Explicit App ID and set its Bundle Identifier to `YOURREVERSEDOMAIN.Fennec.SendTo`. In the App Services section, select *App Groups*.
+5. Create a new App Id. Name it 'Fennec ViewLater'. Give it an Explicit App ID and set its Bundle Identifier to `YOURREVERSEDOMAIN.Fennec.ViewLater`. In the App Services section, select *App Groups*.
+6. For all App Ids that you just created, edit their App Groups and make sure they are all part of the Fennec App Group that you created in step 1.
+
+Now we are going to create three Provisioning Profiles that are linked to the App Ids that we just created:
+
+1. Create a new *Development Provisioning Profile* and link it to the *Fennec* App ID that you created. Select the *Developer Certificates* and *Devices* that you wish to include in this profile. Finally, name this profile *Fennec*.
+2. Create a new *Development Provisioning Profile* and link it to the *Fennec SendTo* App ID that you created. Select the *Developer Certificates* and *Devices* that you wish to include in this profile. Finally, name this profile *Fennec SendTo*.
+3. Create a new *Development Provisioning Profile* and link it to the *Fennec ShareTo* App ID that you created. Select the *Developer Certificates* and *Devices* that you wish to include in this profile. Finally, name this profile *Fennec ShareTo*.
+4. Create a new *Development Provisioning Profile* and link it to the *Fennec ViewLater* App ID that you created. Select the *Developer Certificates* and *Devices* that you wish to include in this profile. Finally, name this profile *Fennec ViewLater*.
+
+Now go to Xcode, *Preferences -> Accounts* and select your developer account. Hit the *View Details* button and then press the little reload button in the bottom left corner. This should sync the Provisioning Profiles and you should see the three profiles appear that you creates earlier.
+
+Almost done. The one thing missing is that we need to adjust the build configuration to use your new bundle identifier.
+
+1. Open Client/Configuration/Fennec.xcconfig
+2. Change MOZ_BUNDLE_ID to `YOURREVERSEDOMAIN`.
+3. Navigate to each of the application targets (Client/SendTo/ShareTo/ViewLater) and your developer account.
+
+Before building, do *Product -> Clean Build Folder* (option-shift-command-k)
+
+You should now be able to build the *Fennec* scheme and run on your device.
+
+We would love a Pull Request for a smarter Xcode project configuration or even a shell script that makes this process simpler.
+
+
+Random notes
+------------
+
+Updating SQLCipher.
+
+As of bug https://bugzilla.mozilla.org/show_bug.cgi?id=1182620 we do not run the SQLCipher 'amalgamation' phase anymore. Instead we have simply included generated copies of `sqlite3.c`, `sqlite3.h` and `sqlite3ext.h` in the project. This works around problems where the amalgamation phase did not work for production builds. It also speeds up things.
+
+To update to a newer version of SQLCipher: check out the original SQLCipher project and build it. Do not copy the project or anything in the Firefox project. Just follow their instructions. Then copy the above three `.c` and `.h` files back into the Firefox project. Also update the `README`, `VERION` and `CHANGELOG` files from the original distribution so that we know what version we have included.

mobile/ios/Docs/Dependency Graphs/README.md

@@ -0,0 +1,21 @@
+
+Available Dependency Graphs
+===============
+
+* `digraph-fxios` - a graph showing dependencies between the different targets in Firefox for iOS
+* `digraph-fxios-carthage` - a graph showing dependencies between Firefox for iOS targets and our Carthage Dependencies
+* `digraph-carthage-fxios` - a graph showing dependencies between our Carthage Dependencies and Firefox for iOS targets
+
+Generating dependency graphs
+-----------
+
+* Download [Graphviz](http://www.graphviz.org/Download_macos.php) or install via homebrew `brew install graphviz`
+* Open Graphviz.app
+* Import `.dot` file
+* Export as `jpg` or `pdf`
+
+
+Examples
+-----------
+
+Here are some [examples](http://graphs.grevian.org/example) of more things you can do with Graphviz

mobile/ios/Docs/Dependency Graphs/dot-files/digraph-carthage-fxios.dot

@@ -0,0 +1,32 @@
+digraph CarthageFxiOS {
+  "Alamofire" -> "Client"
+  "Alamofire" -> "ShareTo"
+  "Alamofire" -> "Shared"
+  "Alamofire" -> "Account"
+  "Alamofire" -> "Sync"
+  "Alamofire" -> "ReadingList"
+  "Snapkit" -> "Client"
+  "Snapkit" -> "Today"
+  "SDWebImage" -> "Client"
+  "SDWebImage" -> "SendTo"
+  "SDWebImage" -> "ShareTo"
+  "SDWebImage" -> "ViewLater"
+  "GCDWebServer" -> "Client"
+  "Adjust" -> "Client"
+  "OnePasswordExtension" -> "Client"
+  "libPhoneNumber-iOS" -> "Client"
+  "web_metadata_ios" -> "Client"
+  "XCGLogger" -> "Client"
+  "XCGLogger" -> "Shared"
+  "XCGLogger" -> "Storage"
+  "XCGLogger" -> "Account"
+  "Fuzi" -> "Client"
+  "Fuzi" -> "SendTo"
+  "Fuzi" -> "ShareTo"
+  "Onyx" -> "Client"
+  "readability" -> "Client"
+  "SwiftKeychainWrapper" -> "Shared"
+  "Deferred" -> "Shared"
+  "KIF" -> "ReadingList"
+  "SQLite" -> "ReadingList"
+}

mobile/ios/Docs/Dependency Graphs/dot-files/digraph-fxios-carthage.dot

@@ -0,0 +1,26 @@
+digraph FxiOSCarthage {
+  "Client" -> "Alamofire"
+  "Client" -> "Snapkit"
+  "Client" -> "SDWebImage"
+  "Client" -> "GCDWebServer"
+  "Client" -> "Adjust"
+  "Client" -> "OnePasswordExtension"
+  "Client" -> "libPhoneNumber-iOS"
+  "Client" -> "web_metadata_ios"
+  "Client" -> "XCGLogger"
+  "Client" -> "Fuzi"
+  "Client" -> "Onyx"
+  "Client" -> "readability"
+  "ShareTo" -> "Alamofire"
+  "ShareTo" -> "SDWebImage"
+  "ShareTo" -> "Fuzi"
+  "Shared" -> "Alamofire"
+  "Shared" -> "XCGLogger"
+  "Shared" -> "SwiftKeychainWrapper"
+  "Shared" -> "Deferred"
+  "Today" -> "Snapkit"
+  "SendTo" -> "SDWebImage"
+  "SendTo" -> "Fuzi"
+  "ViewLater" -> "SDWebImage"
+  "Storage" -> "XCGLogger"
+}

mobile/ios/Docs/Dependency Graphs/dot-files/digraph-fxios.dot

@@ -0,0 +1,49 @@
+digraph FxiOS {
+  "Shared" -> "Storage"
+  "Shared" -> "StorageTests"
+  "Shared" -> "Client"
+  "Shared" -> "ClientTests"
+  "Shared" -> "ViewLater"
+  "Shared" -> "ReadingList"
+  "Shared" -> "SendTo"
+  "Shared" -> "ShareTo"
+  "Shared" -> "Account"
+  "Shared" -> "AccountTests"
+  "Shared" -> "Sync"
+  "Shared" -> "SyncTests"
+  "Shared" -> "UITests"
+  "Shared" -> "Today"
+  "FxA" -> "Account"
+  "FxA" -> "AccountTests"
+  "FxA" -> "Sync"
+  "FxA" -> "SyncTests"
+  "Storage" -> "StorageTests"
+  "Storage" -> "Client"
+  "Storage" -> "ClientTests"
+  "Storage" -> "ViewLater"
+  "Storage" -> "SendTo"
+  "Storage" -> "UITests"
+  "Storage" -> "SyncTests"
+  "Storage" -> "Sync"
+  "Storage" -> "ShareTo"
+  "Sync" -> "SyncTests"
+  "Sync" -> "Client"
+  "Sync" -> "ClientTests"
+  "Sync" -> "ShareTo"
+  "Sync" -> "SendTo"
+  "Sync" -> "ViewLater"
+  "ReadingList" -> "ReadingListTests"
+  "ReadingList" -> "Client"
+  "ReadingList" -> "ClientTests"
+  "Account" -> "AccountTests"
+  "Account" -> "Sync"
+  "Account" -> "SyncTests"
+  "Account" -> "Client"
+  "Account" -> "ClientTests"
+  "Client" -> "ClientTests"
+  "Client" -> "UITests"
+  "Client" -> "ShareTo"
+  "Client" -> "SendTo"
+  "Client" -> "ViewLater"
+  "Client" -> "Today"
+}

mobile/ios/Docs/browserdb.md

@@ -0,0 +1,14 @@
+# How to get your browser DB off the device
+
+* Launch Firefox.
+* Open Settings.
+* Scroll down to "Version" and tap it several times.
+* Scroll down and hit "Debug: copy databases to app container".
+* Connect your device via USB.
+* Open Xcode.
+* Window > Devices. Choose your device.
+* Find "Firefox" on the right side.
+* Click the gear icon, and choose "Download Container…". Save it somewhere.
+* After some time, Finder will open focused on an .xcappdata file.
+* Right-click, "Show Package Contents".
+* Navigate to `AppData/Documents`. Zip up `browser.*`.

mobile/ios/Docs/sync.md

@@ -0,0 +1,76 @@
+# Firefox Sync Through The Lens Of Jaded Developers
+## Or, Let's Use A State Machine For Our Own Sakes
+
+(These docs are approximately one sentence per line to make changes more civilized in version control.)
+
+Sync moves through a number of states.
+
+Initially, we know only the Married account state: we have a valid, verified FxA. This also gives us kB.
+
+Sync will need to know when kB changes, but it does so implicitly -- when kB changes, we fetch a new token, and the X-Client-State header we supply will result in a new storage endpoint.
+
+We transition from this state, then, to one where we also have a Sync storage token and a node assignment. This is the point at which we can first talk to the Sync server. We call this "Initial Assigned". We start in this state when first signed in, and also when creating an account on the device.
+
+(Note that the Sync server allows us to fetch info/collections with an expired token. This is to make the process of short-circuiting a sync cheaper. We still need a server assignment, of course. That we can do this implies that the Sync client has a pull-based relationship with the token server client.)
+
+
+The process thereafter depends on some amount of locally persisted state.
+
+This state falls into three categories:
+
+* Real storage metadata on which our syncing behavior relies. This involves timestamps of collection fetches, syncIDs for storage as a whole and for each collection, and lists of engines. (Perhaps more.)
+
+* Protocol-level signaling, such as Retry-After and Backoff headers. These need to persist in order for us to behave correctly.
+
+* Cached data to avoid repeating operations. The client can cache info/collections for the duration of a sync, and meta/global and crypto/keys until they change.
+
+For more on this, see "Persistence", below.
+
+Starting from that "initial assigned" state, we will always fetch info/collections.
+
+If we have local meta/global and crypto/keys caches, and nothing is indicated as changed in i/c, we can move directly to Ready.
+
+If either is missing, each needs to be fetched and processed. These two tasks should be independent, but typically a change to meta/global will be followed by fetching keys.
+
+If meta/global is missing, one must be created based on the user's datatype elections, ideally preserved from any previous configuration. For consistency's sake, the rest of storage should be wiped prior to uploading a meta/global; a missing meta/global implies that something is wrong, and it probably went wrong during a wipe or node reassignment.
+
+If crypto/keys is missing, there should be no other data on the server. On a non-first-sync, this is likely to be an error state, requiring recovery. If crypto/keys changed, but the server wasn't wiped and refilled, then HMAC errors will result.
+
+So now we have meta/global and crypto/keys, in one of three ways: cached, fetched, or computed and uploaded. If we don't get to that point, we error out; the intermediate fetching states really aren't exposed. If we had to compute and upload -- a very uncommon situation -- we re-enter the state machine from the beginning to simplify analysis.
+
+We call this state "Ready".
+
+From Ready we can perform a number of operations:
+
+* We can wipe the server.
+
+* We can safely change datatype elections or keys for collections.
+
+* We can perform storage operations on one or more collections.
+
+The first two we'll ignore for now. They're relatively straightforward.
+
+The last is encapsulated in a datatype-specific synchronizer and a `Sync15CollectionClient` that pushes and pulls records.
+
+
+The process of synchronizing is driven by info/collections (indicating remote changes), by meta/global (a change to syncID implies a reset), and by local change indicators.
+
+
+## Invalidation
+
+At any point we might get a 401 in response to a storage request. At this point our token is invalidated; we abort this sync, fetch a new token, and continue. That token might itself point to a new Sync server, which might result in a fresh-start sync.
+
+
+## Persistence
+
+So what do we need to persist? Quite apart from the actual datatype-specific stuff:
+
+* We need to store the syncID and last fetch timestamp for each collection. If the syncID changes, we need to throw away the timestamp.
+* We need to store the list of enabled engines, and keep it in lockstep with meta/global. The same for declined.
+* We need to store our own client ID, and use that when uploading our client record.
+* We need to pay attention to kB; if it changes, e.g., via a password change, then any in-memory copy of the Sync Key should be discarded.
+* We ought to cache meta/global.
+* We ought to cache bulk keys. If i/c changes for crypto, we should redownload and compare. If any collection key changes, then we should act as if the syncID changed. (TODO: should we change the syncID for that collection? Probably.)
+  (If we can't decrypt your bulk keys, probably our kB is now invalid. If it's not, the server is corrupt.)
+* We ought to remember the last info/collections.
+* We should store the last info/collections timestamp, so that an i/c fetch can return 304.