How to make deliver (fastlane) download metadata for multiple targets? - ios

I have an Xcode project with six targets, each target is made to build a separate app. I'm trying to setup fastlane to assist me in publication of these apps.
Fastlane docs suggest using .env files in order to handle multiple targets (you can specify app_identifier, team_name, etc. in different .env files, and then, for instance, call fastlane appstore --env ENV_NAME_HERE). However I can't figure out how to set up deliver properly.
deliver init downloads metadata for one target only by default. I need to download metadata for all my targets to different directories (and then use those directories to upload data, obviously).
deliver download_metadata doesn't accept the --env parameter (my Deliverfile depends on env files). I've tried fastlane deliver --env, but it seems to be just a shorthand for deliver, so it doesn't work either.
I guess I could just manually run deliver with different --metadata_path parameters (and all the other parameters since my Deliverfile is invalid, because it depends on env files), and then later specify directories using Deliverfile + .env files. But since I have Deliverfile and .env files already set up (now I use deliver to upload the binary only), I hoped that there is a better way. Is there?
P.S. This is a large legacy project, so splitting it into six different projects would be great, but it's not an option, unfortunately.

I've been struggling with this as well and setting up the submit is easy using the .env files.
But retrieving the initial data is difficult, but not impossible.
To grab the metadata it ran this command:
fastlane deliver download_metadata -m "./Targets/Release/Metadata" -u "itunes#username" -a "com.example.ios"
And for the screenshots:
fastlane deliver download_screenshots -w "./Targets/Release/Screenshots" -u "itunes#username" -a "com.example.ios"

Adding up to #rckoenes answer:
1) Create an .env.yourEnvName file with this info (as an example):
DLV_METADATA_PATH="../Targets/Your_Target/Metadata"
DLV_ITUNESCONNECT_USERNAME="yourItunesUser#something.com"
DLV_BUNDLE_ID="com.yourCompany.yourTarget"
2) Create a lane like this:
desc "Download metadata"
lane :metadata do
sh('fastlane deliver download_metadata -m "$DLV_METADATA_PATH" -u $DLV_ITUNESCONNECT_USERNAME -a $DLV_BUNDLE_ID')
end
3) Call fastlane like this:
fastlane metadata --env yourEnvName
That way it's a little bit cleaner, and you keep the vars in the .env file.
For automating this call for multiple targets, please refer to: https://docs.fastlane.tools/faqs/#multiple-targets-of-the-same-underlying-app

This is a combination of #rckoenes, #Riddick's answer and this fastlane github issue submission.
I was trying #Riddick's answer to have a cleaner workflow but I couldn't make it work to download metadata. For some reason, it only makes the metadata path folder but no metadata downloaded from iTunesConnect. I did some trial and error and found this line of code from the link above this:
ENV["DELIVER_FORCE_OVERWRITE"] = "1"
added it to the lane and worked!
1) Create an .env.yourEnvName file with this info (as an example):
METADATA_PATH="../Targets/Your_Target/Metadata"
APP_IDENTIFIER="com.yourCompany.yourTarget"
2) Create a lane like this:
desc "Download metadata"
lane :metadata do
ENV["DELIVER_FORCE_OVERWRITE"] = "1" # This is the additional line from Riddick's code
sh "fastlane deliver download_metadata --app_identifier #{ENV['APP_IDENTIFIER'] --metadata_path #{ENV['METADATA_PATH']}"
end
3) Call fastlane like this:
fastlane metadata --env yourEnvName
***I did not use the username parameter because I had it on my Deliver File.

Related

Fastlane: How to load .env file from parent directory according to --env

I know that Fastlane automatically load variables from .env, .env.default and .env.{environment}, where environment is supplied by the flag --env in the fastlane command.
In my Fastfile, I need fastlane to load my environment files, that are located in the parent directory.
I want to keep the behavior of loading of loading the .env.{environment} file, when {environment} changes according to what I pass in the --env flag.
This one worked for me:
Using the following before_all inside my platform block:
before_all do |lane|
Dotenv.overload '../../.env'
environment = lane_context[SharedValues::ENVIRONMENT]
unless environment.nil?
puts "Load .env file of #{environment}"
Dotenv.overload '../../.env.' + environment
end
end
(Important!) Putting a blank .env file in the fastlane directory.
The reason to create an empty .env is documented in my empty .env file:
TL;DR: do NOT delete this empty file
This file is blank to make Fastlane define the SharedValue::ENVIRONMENT
variable, which is part of our fastlane/Fastfile configurations.
As you can see in Fastlane's cli_tools,
Fastlane removes the --env index from ARGV pretty early
so we can't know what the user passed to the --env parameter.
Unfortunatly, Fastlane search the .env files only in the fastlane folder
and in it's parent folder (the ios folder, in our case). It means that,
in our project, Fastlane won't find any .env files. (Source)
When Fastlane fails to find .env files, it does not call the function
load_dot_envs_from, which is responsible to define the
SharedValue::ENVIRONMENT variable, which we use inside our Fastfile
(Source)
This file is a hack that will make Fastlane find an empty .env file.
It will then set the SharedValue::ENVIRONMENT to the ARGV value.
We then use SharedValue::ENVIRONMENT inside Fastlane to load the right
file from the right place.

Automating 2FA using Fastlane/CircleCI

What is the correct way to setup 2FA for beta uploads to iTunesConnect/TestFlight?
There are so many links and forums answers but none solve the issue.
Currently I have added the environment variables into CircleCI, including the Application Specific Password generated on the AppleID.
I have a lane in Fastlane that looks like this
desc "Alpha build"
lane :alpha do
match(type: "adhoc")
gym(export_method: "ad-hoc")
upload_to_testflight(skip_submission: true)
end
I run the preauth command below before the alpha lane
- run:
name: Spaceship pre-auth for 2FA
command: bundle exec fastlane spaceauth -u [redacted].com
Fastlane seems to be failing on auth even if the password is correct
Please check your credentials and try again.
This could be an issue with App Store Connect,
Please try unsetting the FASTLANE_SESSION environment variable
If I remove the spaceauth command circleci is failing by timing out waiting for 2FA.
The Fastlane site says that there is no need to use spaceauth unless additional app store connect APIs are being used such as uploading metadata etc. It states that for uploads to testflight only the Application Specific Password should be enough, although this doesnt work either.
Has anyone solved this issue that can advise please?
You can authenticate with Apple through API key. You can generate the key here: https://appstoreconnect.apple.com/access/api.
Once you have the key.p8 file, you can used to auth with the fastlane command: app_store_connect_api_key as follow:
app_store_connect_api_key(
key_id: "ABCDEFG",
issuer_id: "Your_issuer_id",
key_content: File.read("./key.p8").chomp,
duration: 1200,
in_house: false
)
After the execution of the command, the key session is store in the following env var: APP_STORE_CONNECT_API_KEY so your upload to testflight should look like this:
upload_to_testflight(
groups: ["Friends & Family","Mytest-Group"],
ipa: "./build/myapp.ipa",
api_key: Actions.lane_context[SharedValues::APP_STORE_CONNECT_API_KEY]
)
Use CI server's REST API to update the value of FASTLANE_SESSION configuration parameter. In our case we have it defined in one place and it gets reused by all jobs that need to auth to Dev Center / App Store Connect. I haven't looked into details, but I'm sure there's a way to update the job/project parameter via REST call. Then again, have a scheduled job that runs spaceauth and uses REST API to set the new value.
The docs are wrong there - and it's my fault. I updated the docs article before the actual code enabling this is merged.
Your best best currently is to create a second account that doesn't have 2FA enabled, or using spaceauth locally and then copying the returned value into a ENV variable on your CI provider, although it probably will only work 24 hours. We are currently investigating how to improve this.

How to change $(PRODUCT_BUNDLE_IDENTIFIER) in Xcode?

I am build different flavor of Flutter app with different Firebase environment (development and production). I need set different bundle ID for development and production in Xcode for iOS apps.
I am use schemes to configure the different flavor (in Build Settings I add environment value for every configuration).
But I have big issue with change $(PRODUCT_BUNDLE_IDENTIFIER). I need add suffix .development to normal app id for development app id.
I have try follow this method(use User Defined Settings) and change info.plist to get variable from User Defined Settings but it not work.
Error is:
The operation couldn’t be completed. Application
“$(EXAMPLE_BUNDLE_ID)" is unknown to FrontBoard.
So it seem when pass in User Defined Setting it is not interpolate correct.
I have also try mix method of add default PRODUCT_BUNDLE_IDENTIFIER and User Defined Settings. For example: com.example.app$(EXAMPLE_BUNDLE_ID) where EXAMPLE_BUNDLE_ID = .development
I also try reference User Defined Setting $(EXAMPLE_BUNDLE_ID) by direct add it to Bundle Identifier in Target General tab under ‘Identity’. But this then change to : -- EXAMPLE_BUNDLE_ID-
I have also try in info.plist use $(PRODUCT_BUNDLE_IDENTIFIER)$(EXAMPLE_BUNDLE_ID) for Bundle Identifier value. But this give similar error:
The operation couldn’t be completed. Application
“com.example.app$(EXAMPLE_BUNDLE_ID)" is unknown to FrontBoard.
Again this look like interpolation issue.
Anyone know solution? I have look but cannot find answer.
This easy for android because just use applicationIdSuffix ".development” in productFlavors. But I cannot find way like this for Xcode.
Do you need to have different package name (Android) and bundle id (iOS) because you need to use Firebase Auth plugin?
In this case for iOS project you shold consider using PlistBuddy and you could set it adding a Run Script in your XCode build phases like that
if [ "${CONFIGURATION}" = "Debug" ]; then
/usr/libexec/PlistBuddy -c "Set :CFBundleIdentifier com.example.developmento.appName" "$PROJECT_DIR/Runner/Info.plist"
echo "Changed bundle id for developement $PROJECT_DIR/Runner/Info.plist"
else
echo "Nothing to do"
fi
Anyway if you don't use Firebase Auth, you can have the same bundle id in different firebase projects.
If you need then to differenziate firebase projects file between staging and production, you could have a look here:
How to choose between development and production firebase project based on build flavours?
UPDATE
So following OP chat, knowing that he's following this tutorial to setup flutter flavors I've tryed myself to see where we were stuck.
Starting point is the following:
Two Firebase project
Use of Firebase Auth module (so the need to change the bundle id between projects)
And of course two different GoogleService-Info.plist
I start with Xcode bundle id and GoogleService-Info.plist set to production (just an option)
Then I've save both GoogleServices-Info-staging.plist and GoogleServices-Info-production.plist save in my ios/Runner folder
Then I setup this build script before the script for Compile Sources
# Type a script or drag a script file from your workspace to insert its path.
if [ "${CONFIGURATION}" == "Debug" ] || [ "${CONFIGURATION}" == "Debug-Runner-staging" ]; then
echo "Setting up staging firebase environment"
/usr/libexec/PlistBuddy -c "Set :CFBundleIdentifier com.example.staging.flutterAppAuthFlavours" "${PROJECT_DIR}/Runner/Info.plist"
cp -r "${PROJECT_DIR}/Runner/GoogleService-Info-staging.plist" "${PROJECT_DIR}/Runner/GoogleService-Info.plist"
echo "$(date) staging flavour - Configuration: ${CONFIGURATION}" > "${PROJECT_DIR}/environment.txt"
elif [ "${CONFIGURATION}" == "Debug-Runner-production" ]; then
echo "Setting up production firebase environment"
/usr/libexec/PlistBuddy -c "Set :CFBundleIdentifier com.example.flutterAppAuthFlavours" "${PROJECT_DIR}/Runner/Info.plist"
cp -r "${PROJECT_DIR}/Runner/GoogleService-Info-production.plist" "${PROJECT_DIR}/Runner/GoogleService-Info.plist"
echo "$(date) production flavour - Configuration: ${CONFIGURATION}" > "${PROJECT_DIR}/environment.txt"
fi
And I called it Setup Firebase Environment (you can call it what you want)
This script store also some logs (with timestamp) in a file called environment.txt inside ios folder in order to easy check what xcode build has done
And now about Schemes and Build Configurations:
I've done two Build Configuration that are the exact copy of my Debug Build Configuration and I called them
Debug-Runner-staging
Debug-Runner-production
The rule of thumb is to name the build configurations as 'Debug-<your flavor>' and you need to have a scheme for every flavors you have, so I have these:
Runner-staging whose Run calls Debug-Runner-staging build configuration
Runner-production whose Run calls Debug-Runner-production build configuration
So now if I call flutter run --flavor Debug-staging I have a build that runs on my staging firebase project.
and if I call flutter run --flavor Debug-production I have a build that runs on my production firebase project.
UPDATE 2
Just for completness you could change bundle id also here:
Anyway it seems that there's a strange behavior that once you build a flavour a second time flutter command build correctly the flavor but run the previos build flavor.
As building with XCode and switching with schemes all works as expected (even the run of the right application) I guess that this could be a flutter command issue. So I suggest you trying file an issue here linking also this SO question/answer.
UPDATE 3
After a bit of intel I've found that flutter tools set the applicaiton launching environment before building the project. So when we change CFBundleIdentifier inside Info.plist the first time, the second time we launch flutter run it takes the previous modified value and try launching this bundle id while during build we are changing it because we are building a different variant.
A possible solution could be to launch a script that change the CFBundleIdentifier inside Info.plist before calling fluetter run.
For example starting with a Info.plist with a production bundle id of com.example.flutterAppAuthFlavours we could do something like that
Here I’ve used sed command just to think different, but you could call always our belowed PlistBuddy to make the change before calling flutter run.

How to change the screenshot path in Rails 5.1 system test

Using Rails 5.1.2
Creating a system test and using the take_screenshot method.
How do i change the location these screenshots are created at?
Looks like the image path is hardcoded in, so you won't be able to change it currently. Probably wouldn't be too difficult to change if you wanted to open an issue over there or create a pull request for them.
If you want to do this on CI, here's the solution I came up with. In my setup I already had a "test-runner.sh" script, with the rspec invocation at the end. There's probably also some sort of after_script: setting in the yml config also available, but I didn't look into it.
rspec ......
status=$?
# /tmp/test-results is where CircleCI looks for "artifacts" which it makes
# available for download after a test run
[ -d "tmp/screenshots" ] && cp -a tmp/screenshots /tmp/test-results/
exit $status

How to specify different .dockerignore files for different builds in the same project?

I used to list the tests directory in .dockerignore so that it wouldn't get included in the image, which I used to run a web service.
Now I'm trying to use Docker to run my unit tests, and in this case I want the tests directory included.
I've checked docker build -h and found no option related.
How can I do this?
Docker 19.03 shipped a solution for this.
The Docker client tries to load <dockerfile-name>.dockerignore first and then falls back to .dockerignore if it can't be found. So docker build -f Dockerfile.foo . first tries to load Dockerfile.foo.dockerignore.
Setting the DOCKER_BUILDKIT=1 environment variable is currently required to use this feature. This flag can be used with docker compose since 1.25.0-rc3 by also specifying COMPOSE_DOCKER_CLI_BUILD=1.
See also comment0, comment1, comment2
from Mugen comment, please note
the custom dockerignore should be in the same directory as the Dockerfile and not in root context directory like the original .dockerignore
i.e.
when calling
DOCKER_BUILDKIT=1
docker build -f /path/to/custom.Dockerfile ...
your .dockerignore file should be at
/path/to/custom.Dockerfile.dockerignore
At the moment, there is no way to do this. There is a lengthy discussion about adding an --ignore flag to Docker to provide the ignore file to use - please see here.
The options you have at the moment are mostly ugly:
Split your project into subdirectories that each have their own Dockerfile and .dockerignore, which might not work in your case.
Create a script that copies the relevant files into a temporary directory and run the Docker build there.
Adding the cleaned tests as a volume mount to the container could be an option here. After you build the image, if running it for testing, mount the source code containing the tests on top of the cleaned up code.
services:
tests:
image: my-clean-image
volumes:
- '../app:/opt/app' # Add removed tests
I've tried activating the DOCKER_BUILDKIT as suggested by #thisismydesign, but I ran into other problems (outside the scope of this question).
As an alternative, I'm creating an intermediary tar by using the -T flag which takes a txt file containing the files to be included in my tar, so it's not so different than a whitelist .dockerignore.
I export this tar and pipe it to the docker build command, and specify my docker file, which can live anywhere in my file hierarchy. In the end it looks like this:
tar -czh -T files-to-include.txt | docker build -f path/to/Dockerfile -
Another option is to have a further build process that includes the tests. The way I do it is this:
If the tests are unit tests then I create a new Docker image that is derived from the main project image; I just stick a FROM at the top, and then ADD the tests, plus any required tools (in my case, mocha, chai and so on). This new 'testing' image now contains both the tests and the original source to be tested. It can then simply be run as is or it can be run in 'watch mode' with volumes mapped to your source and test directories on the host.
If the tests are integration tests--for example the primary image might be a GraphQL server--then the image I create is self-contained, i.e., is not derived from the primary image (it still contains the tests and tools, of course). My tests use environment variables to tell them where to find the endpoint that needs testing, and it's easy enough to get Docker Compose to bring up both a container using the primary image, and another container using the integration testing image, and set the environment variables so that the test suite knows what to test.
Sadly it isn't currently possible to point to a specific file to use for .dockerignore, so we generate it in our build script based on the target/platform/image. As a docker enthusiast it's a sad and embarrassing workaround.

Resources